> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/BerriAI/litellm/llms.txt
> Use this file to discover all available pages before exploring further.

# Provider Overview

> Call 100+ LLMs using the OpenAI format with LiteLLM

## What are Providers?

LiteLLM provides a unified interface to call 100+ LLM providers using the OpenAI format. Instead of learning each provider's unique API, you can use the same code structure across all providers.

<CardGroup cols={2}>
  <Card title="Unified Interface" icon="code">
    Use the same `completion()` function for all providers - just change the model name prefix
  </Card>

  <Card title="OpenAI Format" icon="openai">
    All responses follow OpenAI's format, making it easy to switch between providers
  </Card>

  <Card title="100+ Providers" icon="layer-group">
    Access models from OpenAI, Anthropic, AWS, Google, Azure, and many more
  </Card>

  <Card title="Provider Features" icon="sparkles">
    Streaming, function calling, vision, embeddings - all standardized across providers
  </Card>
</CardGroup>

## Quick Start

Here's how easy it is to use different providers:

<CodeGroup>
  ```python OpenAI theme={null}
  from litellm import completion
  import os

  os.environ["OPENAI_API_KEY"] = "your-api-key"

  response = completion(
      model="openai/gpt-4o",
      messages=[{"role": "user", "content": "Hello!"}]
  )
  ```

  ```python Anthropic theme={null}
  from litellm import completion
  import os

  os.environ["ANTHROPIC_API_KEY"] = "your-api-key"

  response = completion(
      model="anthropic/claude-sonnet-4-20250514",
      messages=[{"role": "user", "content": "Hello!"}]
  )
  ```

  ```python AWS Bedrock theme={null}
  from litellm import completion
  import os

  os.environ["AWS_ACCESS_KEY_ID"] = "your-access-key"
  os.environ["AWS_SECRET_ACCESS_KEY"] = "your-secret-key"
  os.environ["AWS_REGION_NAME"] = "us-east-1"

  response = completion(
      model="bedrock/anthropic.claude-3-5-sonnet-20240620-v1:0",
      messages=[{"role": "user", "content": "Hello!"}]
  )
  ```

  ```python Google Vertex AI theme={null}
  from litellm import completion
  import os

  os.environ["VERTEX_PROJECT"] = "your-project-id"
  os.environ["VERTEX_LOCATION"] = "us-central1"

  response = completion(
      model="vertex_ai/gemini-2.0-flash-exp",
      messages=[{"role": "user", "content": "Hello!"}]
  )
  ```
</CodeGroup>

## Supported Endpoints

LiteLLM standardizes access to multiple endpoint types:

| Endpoint                | Description        | Supported Providers                                              |
| ----------------------- | ------------------ | ---------------------------------------------------------------- |
| `/chat/completions`     | Text generation    | 100+ providers                                                   |
| `/embeddings`           | Text embeddings    | OpenAI, Azure, Bedrock, Cohere, Vertex AI, HuggingFace, and more |
| `/images/generations`   | Image generation   | OpenAI, Azure, Vertex AI, Bedrock, and more                      |
| `/audio/transcriptions` | Speech-to-text     | OpenAI, Azure, Groq, Deepgram                                    |
| `/audio/speech`         | Text-to-speech     | OpenAI, Azure, ElevenLabs                                        |
| `/moderations`          | Content moderation | OpenAI, Azure                                                    |
| `/batches`              | Batch processing   | OpenAI, Azure, Anthropic, Bedrock                                |
| `/rerank`               | Document reranking | Cohere, HuggingFace, Bedrock                                     |

## Provider Categories

<Tabs>
  <Tab title="Major Cloud Providers">
    <CardGroup cols={2}>
      <Card title="OpenAI" icon="openai" href="/providers/openai">
        GPT-4o, GPT-4o-mini, O1, O3-mini, and more
      </Card>

      <Card title="Anthropic" icon="anthropic" href="/providers/anthropic">
        Claude 4.6, Claude 3.7, Claude 3.5 Sonnet
      </Card>

      <Card title="AWS Bedrock" icon="aws" href="/providers/bedrock">
        Claude, Llama, Mistral, Nova, and more on AWS
      </Card>

      <Card title="Google Vertex AI" icon="google" href="/providers/vertex-ai">
        Gemini 2.0, Gemini 1.5 Pro/Flash on Google Cloud
      </Card>

      <Card title="Azure OpenAI" icon="microsoft" href="/providers/azure">
        GPT-4, GPT-3.5, and more on Azure
      </Card>
    </CardGroup>
  </Tab>

  <Tab title="Specialized Providers">
    <CardGroup cols={2}>
      <Card title="Cohere" icon="c" href="/providers/cohere">
        Command R+, Command Light, embeddings, rerank
      </Card>

      <Card title="Groq" icon="microchip" href="/providers/groq">
        Ultra-fast inference for Llama, Mixtral, Gemma
      </Card>

      <Card title="HuggingFace" icon="face-smile" href="/providers/huggingface">
        Access 100k+ open-source models
      </Card>

      <Card title="OpenRouter" icon="route" href="/providers/openrouter">
        Access multiple providers through one API
      </Card>
    </CardGroup>
  </Tab>

  <Tab title="Local & Self-Hosted">
    <CardGroup cols={2}>
      <Card title="Ollama" icon="desktop" href="/providers/ollama">
        Run models locally with Ollama
      </Card>

      <Card title="LM Studio" icon="computer">
        Local model hosting with LM Studio
      </Card>

      <Card title="vLLM" icon="server">
        High-performance inference server
      </Card>

      <Card title="Text Generation WebUI" icon="browser">
        Oobabooga's popular local UI
      </Card>
    </CardGroup>
  </Tab>
</Tabs>

## Key Features Across Providers

### Streaming Support

All major providers support streaming responses for real-time output.

```python theme={null}
response = completion(
    model="openai/gpt-4o",
    messages=[{"role": "user", "content": "Write a story"}],
    stream=True
)

for chunk in response:
    print(chunk.choices[0].delta.content, end="")
```

### Function Calling

LiteLLM standardizes function/tool calling across providers.

```python theme={null}
tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get current weather",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {"type": "string"}
            }
        }
    }
}]

response = completion(
    model="anthropic/claude-3-5-sonnet-20240620",
    messages=[{"role": "user", "content": "What's the weather in NYC?"}],
    tools=tools
)
```

### Vision/Multimodal

Send images to vision-capable models using a consistent format.

```python theme={null}
response = completion(
    model="openai/gpt-4o",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "What's in this image?"},
            {"type": "image_url", "image_url": {"url": "https://..."}}
        ]
    }]
)
```

## Provider-Specific Features

While LiteLLM provides a unified interface, each provider has unique capabilities:

| Feature              | Providers                               |
| -------------------- | --------------------------------------- |
| **Prompt Caching**   | Anthropic, Vertex AI                    |
| **JSON Mode**        | OpenAI, Azure, Anthropic, Vertex AI     |
| **Vision Models**    | OpenAI, Anthropic, Vertex AI, Azure     |
| **Batch API**        | OpenAI, Azure, Anthropic, Bedrock       |
| **Reasoning Models** | OpenAI (O1, O3), Anthropic (Claude 4.6) |
| **Computer Use**     | Anthropic Claude                        |
| **Web Search**       | Anthropic Claude, Perplexity            |

## Model Naming Convention

LiteLLM uses a `provider/model-name` format:

```python theme={null}
# Format: provider/model-name
"openai/gpt-4o"              # OpenAI GPT-4o
"anthropic/claude-3-5-sonnet-20240620"  # Anthropic Claude
"bedrock/anthropic.claude-v2"  # AWS Bedrock Claude
"vertex_ai/gemini-2.0-flash-exp"  # Google Vertex AI Gemini
"azure/gpt-4"                # Azure OpenAI GPT-4
"groq/llama3-70b-8192"       # Groq Llama 3
"ollama/llama3"              # Local Ollama
```

## Authentication

Each provider has its own authentication method:

<Tabs>
  <Tab title="API Keys">
    Most providers use API keys set via environment variables:

    ```bash theme={null}
    export OPENAI_API_KEY="sk-..."
    export ANTHROPIC_API_KEY="sk-ant-..."
    export COHERE_API_KEY="..."
    export GROQ_API_KEY="gsk_..."
    ```
  </Tab>

  <Tab title="Cloud Credentials">
    Cloud providers use their native authentication:

    ```bash theme={null}
    # AWS Bedrock
    export AWS_ACCESS_KEY_ID="..."
    export AWS_SECRET_ACCESS_KEY="..."
    export AWS_REGION_NAME="us-east-1"

    # Google Vertex AI
    export VERTEX_PROJECT="my-project"
    export VERTEX_LOCATION="us-central1"
    export GOOGLE_APPLICATION_CREDENTIALS="path/to/credentials.json"

    # Azure OpenAI
    export AZURE_API_KEY="..."
    export AZURE_API_BASE="https://...openai.azure.com"
    export AZURE_API_VERSION="2024-02-15-preview"
    ```
  </Tab>

  <Tab title="Local Providers">
    Local providers typically use base URLs:

    ```python theme={null}
    # Ollama
    response = completion(
        model="ollama/llama3",
        api_base="http://localhost:11434"
    )

    # LM Studio
    response = completion(
        model="lm_studio/local-model",
        api_base="http://localhost:1234/v1"
    )
    ```
  </Tab>
</Tabs>

## Error Handling

LiteLLM standardizes error handling across all providers:

```python theme={null}
from litellm import completion
from litellm.exceptions import (
    AuthenticationError,
    RateLimitError,
    ContextWindowExceededError,
    APIError
)

try:
    response = completion(
        model="openai/gpt-4o",
        messages=[{"role": "user", "content": "Hello!"}]
    )
except AuthenticationError as e:
    print(f"Invalid API key: {e}")
except RateLimitError as e:
    print(f"Rate limit exceeded: {e}")
except ContextWindowExceededError as e:
    print(f"Message too long: {e}")
except APIError as e:
    print(f"API error: {e}")
```

## Next Steps

<CardGroup cols={2}>
  <Card title="OpenAI" icon="openai" href="/providers/openai">
    Get started with OpenAI models
  </Card>

  <Card title="Anthropic" icon="anthropic" href="/providers/anthropic">
    Use Claude models with advanced features
  </Card>

  <Card title="Streaming" icon="wave-pulse" href="/providers/streaming">
    Learn about streaming responses
  </Card>

  <Card title="Function Calling" icon="function" href="/providers/function-calling">
    Implement tool and function calling
  </Card>
</CardGroup>

## Additional Resources

* [Complete Provider List](https://docs.litellm.ai/docs/providers) - Full list of 100+ supported providers
* [Model Prices](https://models.litellm.ai/) - Browse all available models with pricing
* [Proxy Server](/proxy/overview) - Use providers through the LiteLLM Gateway
* [Router](/router/overview) - Load balancing and fallbacks across providers