> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/BerriAI/litellm/llms.txt
> Use this file to discover all available pages before exploring further.

# Configuration Schema

> Complete configuration schema for LiteLLM proxy config.yaml

## Overview

The LiteLLM proxy uses a YAML configuration file to define models, routing, authentication, and other settings.

## File Location

Default: `config.yaml`

Start proxy with config:

```bash theme={null}
litellm --config config.yaml
```

## Complete Schema

```yaml theme={null}
model_list:
  - model_name: gpt-4
    litellm_params:
      model: azure/gpt-4
      api_base: https://your-endpoint.openai.azure.com/
      api_key: os.environ/AZURE_API_KEY
      api_version: "2024-02-01"
    model_info:
      mode: chat
      supports_function_calling: true
      supports_vision: true

litellm_settings:
  success_callback: ["langfuse", "lunary"]
  failure_callback: ["sentry"]
  set_verbose: true
  drop_params: true
  max_parallel_requests: 100
  request_timeout: 600
  num_retries: 3
  fallbacks:
    - gpt-4: ["gpt-3.5-turbo", "claude-2"]
  context_window_fallbacks:
    - gpt-3.5-turbo: ["gpt-3.5-turbo-16k"]

general_settings:
  master_key: sk-1234
  database_url: postgresql://...
  store_model_in_db: true
  allowed_routes: ["chat/completions", "embeddings"]
  key_management_settings:
    default_key_duration: 30d
    max_key_duration: 365d

router_settings:
  routing_strategy: latency-based-routing
  routing_strategy_args:
    ttl: 60
  model_group_alias:
    gpt-4: production-gpt-4
  redis_host: localhost
  redis_port: 6379
  redis_password: os.environ/REDIS_PASSWORD
  num_retries: 3
  timeout: 30
  allowed_fails: 3
  cooldown_time: 60
```

## Configuration Sections

### model\_list

Define model deployments for the proxy.

<ParamField path="model_name" type="string" required>
  User-facing model name. Multiple deployments can share the same model\_name for load balancing.
</ParamField>

<ParamField path="litellm_params" type="object" required>
  Parameters passed to litellm.completion().

  <Expandable title="litellm_params fields">
    <ParamField path="model" type="string" required>
      Provider-specific model identifier.

      Examples:

      * `azure/gpt-4`
      * `bedrock/anthropic.claude-v2`
      * `vertex_ai/gemini-pro`
    </ParamField>

    <ParamField path="api_key" type="string">
      API key. Use `os.environ/VAR_NAME` to load from environment.
    </ParamField>

    <ParamField path="api_base" type="string">
      API endpoint base URL.
    </ParamField>

    <ParamField path="api_version" type="string">
      API version (provider-specific).
    </ParamField>

    <ParamField path="timeout" type="number">
      Request timeout in seconds.
    </ParamField>
  </Expandable>
</ParamField>

<ParamField path="model_info" type="object">
  Metadata about the model.

  <Expandable title="model_info fields">
    <ParamField path="mode" type="string">
      Model mode: `"chat"`, `"completion"`, `"embedding"`, `"image_generation"`
    </ParamField>

    <ParamField path="input_cost_per_token" type="number">
      Cost per input token in USD.
    </ParamField>

    <ParamField path="output_cost_per_token" type="number">
      Cost per output token in USD.
    </ParamField>

    <ParamField path="max_tokens" type="number">
      Maximum tokens supported.
    </ParamField>

    <ParamField path="supports_function_calling" type="boolean">
      Whether model supports function calling.
    </ParamField>

    <ParamField path="supports_vision" type="boolean">
      Whether model supports vision/image inputs.
    </ParamField>
  </Expandable>
</ParamField>

<ParamField path="tpm" type="number">
  Tokens per minute limit for this deployment.
</ParamField>

<ParamField path="rpm" type="number">
  Requests per minute limit for this deployment.
</ParamField>

#### Example

```yaml theme={null}
model_list:
  # OpenAI GPT-4
  - model_name: gpt-4
    litellm_params:
      model: gpt-4
      api_key: os.environ/OPENAI_API_KEY
    tpm: 100000
    rpm: 1000
  
  # Azure GPT-4
  - model_name: gpt-4
    litellm_params:
      model: azure/gpt-4
      api_base: https://your-endpoint.openai.azure.com/
      api_key: os.environ/AZURE_API_KEY
      api_version: "2024-02-01"
    tpm: 200000
    rpm: 2000
  
  # Claude 2
  - model_name: claude-2
    litellm_params:
      model: claude-2
      api_key: os.environ/ANTHROPIC_API_KEY
    tpm: 100000
    rpm: 1000
  
  # Bedrock Claude
  - model_name: claude-bedrock
    litellm_params:
      model: bedrock/anthropic.claude-v2
      aws_region_name: us-east-1
  
  # Embedding model
  - model_name: text-embedding-3-small
    litellm_params:
      model: text-embedding-3-small
      api_key: os.environ/OPENAI_API_KEY
```

***

### litellm\_settings

Global LiteLLM configuration.

<ParamField path="success_callback" type="array">
  Callbacks to run on successful requests.

  Supported: `langfuse`, `lunary`, `helicone`, `supabase`, `datadog`, `prometheus`, `custom`
</ParamField>

<ParamField path="failure_callback" type="array">
  Callbacks to run on failed requests.

  Supported: `sentry`, `slack`, `webhook`, `custom`
</ParamField>

<ParamField path="set_verbose" type="boolean" default="false">
  Enable verbose logging.
</ParamField>

<ParamField path="drop_params" type="boolean" default="false">
  Drop unsupported parameters instead of erroring.
</ParamField>

<ParamField path="max_parallel_requests" type="number">
  Maximum parallel requests.
</ParamField>

<ParamField path="request_timeout" type="number" default="600">
  Default request timeout in seconds.
</ParamField>

<ParamField path="num_retries" type="number" default="0">
  Number of retries on failure.
</ParamField>

<ParamField path="fallbacks" type="array">
  Fallback model configurations.

  ```yaml theme={null}
  fallbacks:
    - gpt-4: ["gpt-3.5-turbo", "claude-2"]
    - claude-2: ["gpt-3.5-turbo"]
  ```
</ParamField>

<ParamField path="context_window_fallbacks" type="array">
  Fallbacks for context window exceeded errors.

  ```yaml theme={null}
  context_window_fallbacks:
    - gpt-3.5-turbo: ["gpt-3.5-turbo-16k"]
    - gpt-4: ["gpt-4-32k"]
  ```
</ParamField>

<ParamField path="cache" type="boolean" default="false">
  Enable caching.
</ParamField>

<ParamField path="cache_params" type="object">
  Caching configuration.

  ```yaml theme={null}
  cache_params:
    type: redis
    host: localhost
    port: 6379
    ttl: 3600
  ```
</ParamField>

#### Example

```yaml theme={null}
litellm_settings:
  success_callback: ["langfuse", "prometheus"]
  failure_callback: ["sentry", "slack"]
  set_verbose: true
  drop_params: true
  request_timeout: 300
  num_retries: 3
  fallbacks:
    - gpt-4: ["gpt-3.5-turbo"]
  cache: true
  cache_params:
    type: redis
    host: localhost
    port: 6379
    ttl: 3600
```

***

### general\_settings

Proxy server configuration.

<ParamField path="master_key" type="string">
  Admin master key for proxy management.

  ```yaml theme={null}
  master_key: sk-1234
  # or from environment
  master_key: os.environ/LITELLM_MASTER_KEY
  ```
</ParamField>

<ParamField path="database_url" type="string">
  PostgreSQL connection string for key/user/team storage.

  ```yaml theme={null}
  database_url: postgresql://user:pass@localhost:5432/litellm
  ```
</ParamField>

<ParamField path="store_model_in_db" type="boolean" default="false">
  Store model configurations in database.
</ParamField>

<ParamField path="allowed_routes" type="array">
  Restrict which endpoints are enabled.

  ```yaml theme={null}
  allowed_routes:
    - "chat/completions"
    - "embeddings"
    - "key/generate"
  ```
</ParamField>

<ParamField path="ui_access_mode" type="string" default="admin">
  UI access control.

  Options: `"admin"`, `"all"`
</ParamField>

<ParamField path="max_budget" type="number">
  Global budget limit.
</ParamField>

<ParamField path="budget_duration" type="string">
  Budget reset period: `"1d"`, `"30d"`, etc.
</ParamField>

#### Example

```yaml theme={null}
general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY
  database_url: os.environ/DATABASE_URL
  store_model_in_db: true
  allowed_routes:
    - "chat/completions"
    - "embeddings"
    - "key/generate"
    - "key/list"
    - "team/new"
  ui_access_mode: admin
  max_budget: 10000.0
  budget_duration: 30d
```

***

### router\_settings

Router configuration for load balancing.

<ParamField path="routing_strategy" type="string" default="simple-shuffle">
  Load balancing strategy.

  Options:

  * `simple-shuffle`: Random selection
  * `least-busy`: Fewest ongoing requests
  * `usage-based-routing`: Based on TPM/RPM
  * `latency-based-routing`: Lowest latency
  * `cost-based-routing`: Lowest cost
</ParamField>

<ParamField path="routing_strategy_args" type="object">
  Strategy-specific arguments.

  ```yaml theme={null}
  routing_strategy_args:
    ttl: 60  # For latency-based routing
  ```
</ParamField>

<ParamField path="model_group_alias" type="object">
  Model aliases.

  ```yaml theme={null}
  model_group_alias:
    gpt-4: production-gpt-4
    claude: production-claude
  ```
</ParamField>

<ParamField path="redis_host" type="string">
  Redis host for caching.
</ParamField>

<ParamField path="redis_port" type="number" default="6379">
  Redis port.
</ParamField>

<ParamField path="redis_password" type="string">
  Redis password.
</ParamField>

<ParamField path="num_retries" type="number" default="0">
  Router-level retries.
</ParamField>

<ParamField path="timeout" type="number" default="600">
  Router timeout.
</ParamField>

<ParamField path="allowed_fails" type="number" default="3">
  Failures before cooldown.
</ParamField>

<ParamField path="cooldown_time" type="number" default="60">
  Cooldown duration in seconds.
</ParamField>

#### Example

```yaml theme={null}
router_settings:
  routing_strategy: latency-based-routing
  routing_strategy_args:
    ttl: 60
  model_group_alias:
    gpt-4: prod-gpt-4
  redis_host: localhost
  redis_port: 6379
  redis_password: os.environ/REDIS_PASSWORD
  num_retries: 3
  timeout: 30
  allowed_fails: 5
  cooldown_time: 120
```

## Complete Example

```yaml theme={null}
model_list:
  # GPT-4 with load balancing
  - model_name: gpt-4
    litellm_params:
      model: azure/gpt-4
      api_base: https://endpoint1.openai.azure.com/
      api_key: os.environ/AZURE_API_KEY_1
      api_version: "2024-02-01"
    tpm: 100000
    rpm: 1000
  
  - model_name: gpt-4
    litellm_params:
      model: gpt-4
      api_key: os.environ/OPENAI_API_KEY
    tpm: 90000
    rpm: 900
  
  # GPT-3.5-Turbo
  - model_name: gpt-3.5-turbo
    litellm_params:
      model: gpt-3.5-turbo
      api_key: os.environ/OPENAI_API_KEY
    tpm: 1000000
    rpm: 10000
  
  # Claude
  - model_name: claude-2
    litellm_params:
      model: claude-2
      api_key: os.environ/ANTHROPIC_API_KEY
    tpm: 100000
    rpm: 1000
  
  # Embeddings
  - model_name: text-embedding-3-small
    litellm_params:
      model: text-embedding-3-small
      api_key: os.environ/OPENAI_API_KEY

litellm_settings:
  success_callback: ["langfuse", "prometheus"]
  failure_callback: ["sentry"]
  set_verbose: false
  drop_params: true
  request_timeout: 300
  num_retries: 3
  fallbacks:
    - gpt-4: ["gpt-3.5-turbo", "claude-2"]
  context_window_fallbacks:
    - gpt-3.5-turbo: ["gpt-3.5-turbo-16k"]
  cache: true
  cache_params:
    type: redis
    host: localhost
    port: 6379
    ttl: 3600

general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY
  database_url: os.environ/DATABASE_URL
  store_model_in_db: true
  allowed_routes:
    - "chat/completions"
    - "embeddings"
    - "key/generate"
    - "key/list"
    - "team/new"
    - "user/new"
  ui_access_mode: admin

router_settings:
  routing_strategy: latency-based-routing
  routing_strategy_args:
    ttl: 60
  redis_host: localhost
  redis_port: 6379
  redis_password: os.environ/REDIS_PASSWORD
  num_retries: 3
  timeout: 30
  allowed_fails: 3
  cooldown_time: 60
```

## Environment Variables

Load values from environment:

```yaml theme={null}
model_list:
  - model_name: gpt-4
    litellm_params:
      model: azure/gpt-4
      api_key: os.environ/AZURE_API_KEY  # Loads from $AZURE_API_KEY
      api_base: os.environ/AZURE_API_BASE

general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY
  database_url: os.environ/DATABASE_URL
```

## Validation

Validate your config:

```bash theme={null}
litellm --config config.yaml --test
```

## Related

* [Environment Variables](/api/environment-variables)
* [Router Configuration](/api/router-config)
* [Proxy Setup Guide](https://docs.litellm.ai/docs/proxy/quick_start)