> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/BerriAI/litellm/llms.txt
> Use this file to discover all available pages before exploring further.

# Introduction to LiteLLM

> Call 100+ LLMs using the OpenAI format - Bedrock, Azure, OpenAI, VertexAI, Anthropic, Groq, and more

<img src="https://github.com/user-attachments/assets/c5ee0412-6fb5-4fb6-ab5b-bafae4209ca6" alt="LiteLLM" />

## What is LiteLLM?

LiteLLM is a unified interface for calling 100+ Large Language Models (LLMs) using the OpenAI format. It translates inputs to provider-specific completion, embedding, and image generation endpoints while providing consistent output across all providers.

**Performance**: 8ms P95 latency at 1k RPS

## Key Features

<CardGroup cols={2}>
  <Card title="Python SDK" icon="python" href="/quickstart-sdk">
    Use LiteLLM directly in your Python code with a simple, unified interface
  </Card>

  <Card title="AI Gateway (Proxy)" icon="server" href="/quickstart-proxy">
    Deploy a centralized LLM gateway with authentication, cost tracking, and monitoring
  </Card>

  <Card title="100+ LLM Providers" icon="globe">
    OpenAI, Azure, Anthropic, Vertex AI, Bedrock, Groq, and many more
  </Card>

  <Card title="Router with Fallbacks" icon="arrows-rotate">
    Load balancing, retry logic, and automatic fallbacks across deployments
  </Card>
</CardGroup>

## Use Cases

### LLMs - Call 100+ Models

LiteLLM supports all major completion endpoints including `/chat/completions`, `/responses`, `/embeddings`, `/images`, `/audio`, `/batches`, `/rerank`, `/a2a`, and `/messages`.

**Supported Providers**: OpenAI, Anthropic, Azure, Vertex AI, Bedrock, Groq, Cohere, Mistral, Deepseek, Cerebras, and 90+ more providers.

### Agents - Invoke A2A Agents

Connect to A2A (Agent-to-Agent) protocol agents from multiple providers:

* LangGraph
* Vertex AI Agent Engine
* Azure AI Foundry
* Bedrock AgentCore
* Pydantic AI

### MCP Tools - Connect MCP Servers

Integrate Model Context Protocol (MCP) servers with any LLM:

* Load MCP tools in OpenAI format
* Use with any LiteLLM-supported model
* Compatible with Cursor IDE and other tools

## How to Choose

<table>
  <thead>
    <tr>
      <th />

      <th><strong>Python SDK</strong></th>
      <th><strong>AI Gateway (Proxy)</strong></th>
    </tr>
  </thead>

  <tbody>
    <tr>
      <td><strong>Use Case</strong></td>
      <td>Direct integration in your Python codebase</td>
      <td>Central service (LLM Gateway) to access multiple LLMs</td>
    </tr>

    <tr>
      <td><strong>Who Uses It?</strong></td>
      <td>Developers building LLM projects</td>
      <td>Gen AI Enablement / ML Platform Teams</td>
    </tr>

    <tr>
      <td><strong>Key Features</strong></td>

      <td>
        * Direct Python library integration<br />
        * Router with retry/fallback logic<br />
        * Application-level load balancing<br />
        * Exception handling with OpenAI-compatible errors<br />
        * Observability callbacks (Lunary, MLflow, Langfuse, etc.)
      </td>

      <td>
        * Centralized API gateway with auth<br />
        * Multi-tenant cost tracking per project/user<br />
        * Per-project customization (logging, guardrails, caching)<br />
        * Virtual keys for secure access control<br />
        * Admin dashboard UI for monitoring
      </td>
    </tr>
  </tbody>
</table>

## Quick Start

<CardGroup cols={2}>
  <Card title="Python SDK Quick Start" icon="code" href="/quickstart-sdk">
    Get started with the Python SDK in 2 minutes
  </Card>

  <Card title="Proxy Quick Start" icon="rocket" href="/quickstart-proxy">
    Deploy your AI Gateway in 5 minutes
  </Card>
</CardGroup>

## OSS Adopters

Trusted by leading organizations:

* **Stripe** - Financial infrastructure
* **Netflix** - Content streaming
* **Google ADK** - AI development
* **OpenAI Agents SDK** - Official OpenAI integration
* **OpenHands** - AI coding assistant
* **Greptile** - Code understanding

## Community & Support

<CardGroup cols={3}>
  <Card title="Discord" icon="discord" href="https://discord.gg/wuPM9dRgDw">
    Join our Discord community
  </Card>

  <Card title="Documentation" icon="book" href="https://docs.litellm.ai">
    Full documentation
  </Card>

  <Card title="GitHub" icon="github" href="https://github.com/BerriAI/litellm">
    Star us on GitHub
  </Card>
</CardGroup>

## Next Steps

<Steps>
  <Step title="Choose Your Path">
    Decide whether you want to use the Python SDK or deploy the AI Gateway
  </Step>

  <Step title="Follow Quick Start">
    Complete the relevant quick start guide for your chosen path
  </Step>

  <Step title="Explore Features">
    Learn about advanced features like caching, fallbacks, and observability
  </Step>
</Steps>