> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/BerriAI/litellm/llms.txt
> Use this file to discover all available pages before exploring further.

# Quick Start - AI Gateway (Proxy)

> Deploy your LiteLLM AI Gateway in 5 minutes with authentication, cost tracking, and monitoring

## What is the LiteLLM Proxy?

The LiteLLM Proxy is a centralized AI Gateway that provides:

* **Authentication & Authorization** - Virtual keys for secure access control
* **Cost Tracking** - Per-user, per-project spend monitoring
* **Rate Limiting** - Control usage with TPM/RPM limits
* **Load Balancing** - Distribute requests across multiple deployments
* **Caching** - Reduce costs with intelligent response caching
* **Admin Dashboard** - Web UI for management and monitoring

## Quick Installation

<Steps>
  <Step title="Install LiteLLM with Proxy">
    Install LiteLLM with proxy dependencies:

    ```bash theme={null}
    pip install 'litellm[proxy]'
    ```
  </Step>

  <Step title="Start the Proxy">
    Start the proxy with a single model (OpenAI GPT-4):

    ```bash theme={null}
    # Set your OpenAI API key
    export OPENAI_API_KEY="your-openai-key"

    # Start the proxy
    litellm --model gpt-4o
    ```

    The proxy will start on `http://0.0.0.0:4000`
  </Step>

  <Step title="Test Your Gateway">
    Make your first request using OpenAI SDK:

    ```python theme={null}
    import openai

    client = openai.OpenAI(
        api_key="anything",  # Can be anything when no auth is configured
        base_url="http://0.0.0.0:4000"
    )

    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": "Hello from LiteLLM!"}]
    )

    print(response.choices[0].message.content)
    ```
  </Step>
</Steps>

<Note>
  **Production Deployment**: For production, use the configuration file approach below with authentication enabled.
</Note>

## Configuration File Setup

For production deployments, use a configuration file to define your models and settings.

<Steps>
  <Step title="Create Config File">
    Create a `config.yaml` file:

    ```yaml config.yaml theme={null}
    model_list:
      - model_name: gpt-4
        litellm_params:
          model: openai/gpt-4o
          api_key: os.environ/OPENAI_API_KEY
      
      - model_name: claude-3
        litellm_params:
          model: anthropic/claude-sonnet-4-20250514
          api_key: os.environ/ANTHROPIC_API_KEY
      
      - model_name: azure-gpt-4
        litellm_params:
          model: azure/gpt-4o
          api_key: os.environ/AZURE_API_KEY
          api_base: os.environ/AZURE_API_BASE
          api_version: "2025-02-01-preview"

    # General settings
    general_settings:
      master_key: "sk-1234"  # Change this!
      database_url: "postgresql://user:password@host:5432/dbname"  # Optional
    ```
  </Step>

  <Step title="Set Environment Variables">
    Set your API keys:

    ```bash theme={null}
    export OPENAI_API_KEY="your-openai-key"
    export ANTHROPIC_API_KEY="your-anthropic-key"
    export AZURE_API_KEY="your-azure-key"
    export AZURE_API_BASE="https://your-endpoint.openai.azure.com/"
    ```
  </Step>

  <Step title="Start with Config">
    Start the proxy with your config file:

    ```bash theme={null}
    litellm --config config.yaml --port 4000
    ```
  </Step>
</Steps>

## Create Virtual Keys

Virtual keys provide secure access control with per-key budgets and rate limits.

<CodeGroup>
  ```bash cURL theme={null}
  curl -X POST 'http://0.0.0.0:4000/key/generate' \
    -H 'Authorization: Bearer sk-1234' \
    -H 'Content-Type: application/json' \
    -d '{
      "models": ["gpt-4", "claude-3"],
      "max_budget": 10.0,
      "budget_duration": "30d",
      "tpm_limit": 100000,
      "rpm_limit": 100,
      "metadata": {"user": "john@company.com"}
    }'
  ```

  ```python Python theme={null}
  import requests

  response = requests.post(
      "http://0.0.0.0:4000/key/generate",
      headers={"Authorization": "Bearer sk-1234"},
      json={
          "models": ["gpt-4", "claude-3"],
          "max_budget": 10.0,
          "budget_duration": "30d",
          "tpm_limit": 100000,
          "rpm_limit": 100,
          "metadata": {"user": "john@company.com"}
      }
  )

  key_data = response.json()
  print(f"Virtual Key: {key_data['key']}")
  ```
</CodeGroup>

## Use Virtual Keys

Use the generated virtual key to make requests:

```python theme={null}
import openai

client = openai.OpenAI(
    api_key="sk-your-generated-virtual-key",
    base_url="http://0.0.0.0:4000"
)

response = client.chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "Hello!"}]
)
```

## Load Balancing

Distribute requests across multiple deployments of the same model:

```yaml config.yaml theme={null}
model_list:
  # Multiple deployments of the same model
  - model_name: gpt-4
    litellm_params:
      model: openai/gpt-4o
      api_key: os.environ/OPENAI_API_KEY_1
  
  - model_name: gpt-4
    litellm_params:
      model: openai/gpt-4o
      api_key: os.environ/OPENAI_API_KEY_2
  
  - model_name: gpt-4
    litellm_params:
      model: azure/gpt-4o
      api_key: os.environ/AZURE_API_KEY
      api_base: os.environ/AZURE_API_BASE

router_settings:
  routing_strategy: "least-busy"  # or "simple-shuffle", "latency-based-routing"
  num_retries: 2
```

## Fallbacks

Automatically fallback to alternative models on failure:

```yaml config.yaml theme={null}
model_list:
  - model_name: gpt-4
    litellm_params:
      model: openai/gpt-4o
      api_key: os.environ/OPENAI_API_KEY
  
  - model_name: claude-fallback
    litellm_params:
      model: anthropic/claude-sonnet-4-20250514
      api_key: os.environ/ANTHROPIC_API_KEY

router_settings:
  fallbacks:
    - gpt-4: ["claude-fallback"]
  num_retries: 2
```

## Caching

Enable caching to reduce costs and improve response times:

```yaml config.yaml theme={null}
model_list:
  - model_name: gpt-4
    litellm_params:
      model: openai/gpt-4o
      api_key: os.environ/OPENAI_API_KEY

general_settings:
  cache: true
  cache_params:
    type: "redis"
    host: "localhost"
    port: 6379
    ttl: 600  # Cache for 10 minutes
```

## Admin Dashboard

Access the web-based admin dashboard to:

* Create and manage virtual keys
* Monitor usage and costs
* View request logs
* Configure models and settings

<Steps>
  <Step title="Access Dashboard">
    Open your browser and navigate to:

    ```
    http://0.0.0.0:4000/ui
    ```
  </Step>

  <Step title="Login">
    Login with your master key:

    * Master Key: `sk-1234` (or the value from your config)
  </Step>

  <Step title="Explore Features">
    * **Keys**: Create and manage virtual keys
    * **Models**: View and configure available models
    * **Usage**: Monitor costs and request metrics
    * **Logs**: View detailed request logs
  </Step>
</Steps>

## Docker Deployment

Deploy using Docker for production:

<CodeGroup>
  ```bash Docker Compose theme={null}
  # docker-compose.yml
  version: '3.8'

  services:
    litellm:
      image: ghcr.io/berriai/litellm:main-stable
      ports:
        - "4000:4000"
      environment:
        - OPENAI_API_KEY=${OPENAI_API_KEY}
        - ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}
        - DATABASE_URL=${DATABASE_URL}
      volumes:
        - ./config.yaml:/app/config.yaml
      command: ["--config", "/app/config.yaml", "--port", "4000"]
    
    postgres:
      image: postgres:15-alpine
      environment:
        - POSTGRES_DB=litellm
        - POSTGRES_USER=litellm
        - POSTGRES_PASSWORD=your-secure-password
      volumes:
        - postgres_data:/var/lib/postgresql/data

  volumes:
    postgres_data:
  ```

  ```bash Docker Run theme={null}
  docker run -d \
    -p 4000:4000 \
    -e OPENAI_API_KEY=$OPENAI_API_KEY \
    -e ANTHROPIC_API_KEY=$ANTHROPIC_API_KEY \
    -v $(pwd)/config.yaml:/app/config.yaml \
    ghcr.io/berriai/litellm:main-stable \
    --config /app/config.yaml --port 4000
  ```
</CodeGroup>

## Observability

Integrate with observability platforms:

```yaml config.yaml theme={null}
model_list:
  - model_name: gpt-4
    litellm_params:
      model: openai/gpt-4o
      api_key: os.environ/OPENAI_API_KEY

litellm_settings:
  success_callback: ["langfuse", "prometheus", "datadog"]
  
  # Langfuse configuration
  langfuse_public_key: os.environ/LANGFUSE_PUBLIC_KEY
  langfuse_secret_key: os.environ/LANGFUSE_SECRET_KEY
  langfuse_host: "https://cloud.langfuse.com"
  
  # Prometheus configuration
  prometheus: true
  
  # Datadog configuration
  datadog_api_key: os.environ/DATADOG_API_KEY
  datadog_site: "datadoghq.com"
```

## API Endpoints

The proxy exposes OpenAI-compatible endpoints:

<CardGroup cols={2}>
  <Card title="Chat Completions" icon="message">
    `POST /chat/completions`
  </Card>

  <Card title="Completions" icon="pen">
    `POST /completions`
  </Card>

  <Card title="Embeddings" icon="brain">
    `POST /embeddings`
  </Card>

  <Card title="Images" icon="image">
    `POST /images/generations`
  </Card>

  <Card title="Audio" icon="microphone">
    `POST /audio/transcriptions`
  </Card>

  <Card title="Models" icon="list">
    `GET /models`
  </Card>
</CardGroup>

## Environment Variables

Common environment variables:

```bash theme={null}
# API Keys
export OPENAI_API_KEY="your-key"
export ANTHROPIC_API_KEY="your-key"
export AZURE_API_KEY="your-key"

# Database (Optional)
export DATABASE_URL="postgresql://user:password@host:5432/dbname"

# Redis (Optional)
export REDIS_HOST="localhost"
export REDIS_PORT="6379"

# Proxy Settings
export LITELLM_MASTER_KEY="sk-1234"
export LITELLM_PORT="4000"
```

## What's Next?

<CardGroup cols={2}>
  <Card title="Authentication" icon="lock">
    Set up SSO, LDAP, or custom authentication
  </Card>

  <Card title="Guardrails" icon="shield">
    Add content moderation and safety guardrails
  </Card>

  <Card title="Teams & Projects" icon="users">
    Organize users into teams with separate budgets
  </Card>

  <Card title="Enterprise Features" icon="building">
    Explore enterprise features like SSO and SLAs
  </Card>
</CardGroup>

<Warning>
  **Security Best Practices**

  * Always change the default `master_key`
  * Use environment variables for sensitive data
  * Enable HTTPS in production
  * Use a PostgreSQL database for persistence
  * Regularly rotate API keys
</Warning>

<Note>
  **Need Help?** Join our [Discord community](https://discord.gg/wuPM9dRgDw) or check out the [full documentation](https://docs.litellm.ai).
</Note>