> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/BerriAI/litellm/llms.txt
> Use this file to discover all available pages before exploring further.
# Chat Completion API
> Generate chat completions using any LLM provider through a unified API
## Overview
The `completion()` function provides a unified interface to call 100+ LLM providers. It translates OpenAI-format requests to provider-specific formats and returns standardized responses.
## Basic Usage
```python theme={null}
from litellm import completion
response = completion(
model="gpt-3.5-turbo",
messages=[{"role": "user", "content": "Hello, how are you?"}]
)
print(response.choices[0].message.content)
```
## Function Signature
```python theme={null}
def completion(
model: str,
messages: List[Dict[str, str]],
# Optional OpenAI params
functions: Optional[List] = None,
function_call: Optional[str] = None,
timeout: Optional[Union[float, str, httpx.Timeout]] = None,
temperature: Optional[float] = None,
top_p: Optional[float] = None,
n: Optional[int] = None,
stream: Optional[bool] = None,
stream_options: Optional[dict] = None,
stop: Optional[Union[str, List[str]]] = None,
max_tokens: Optional[int] = None,
max_completion_tokens: Optional[int] = None,
presence_penalty: Optional[float] = None,
frequency_penalty: Optional[float] = None,
logit_bias: Optional[dict] = None,
user: Optional[str] = None,
# OpenAI v1.0+ params
response_format: Optional[Union[dict, Type[BaseModel]]] = None,
seed: Optional[int] = None,
tools: Optional[List] = None,
tool_choice: Optional[Union[str, dict]] = None,
parallel_tool_calls: Optional[bool] = None,
logprobs: Optional[bool] = None,
top_logprobs: Optional[int] = None,
reasoning_effort: Optional[str] = None,
# API configuration
base_url: Optional[str] = None,
api_version: Optional[str] = None,
api_key: Optional[str] = None,
extra_headers: Optional[dict] = None,
# LiteLLM params
custom_llm_provider: Optional[str] = None,
**kwargs
) -> Union[ModelResponse, CustomStreamWrapper]
```
## Parameters
The model to use for completion. Examples: `gpt-4`, `claude-3-5-sonnet-20241022`, `gemini-pro`
List of messages in the conversation. Each message should have `role` and `content` fields.
```python theme={null}
messages = [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "What is the capital of France?"}
]
```
### Optional Parameters
Controls randomness in the output (0.0 to 2.0). Lower values make output more focused and deterministic.
Maximum number of tokens to generate in the completion.
If `True`, returns a streaming response. Default: `False`
List of tools (functions) the model can call. See [Function Calling](/sdk/function-calling) for details.
Specify the output format. Can be a dict with `{"type": "json_object"}` or a Pydantic model.
Request timeout in seconds. Default: 600 (10 minutes)
API key for the provider. If not provided, reads from environment variables.
Custom API base URL for the provider.
## Response Format
The function returns a `ModelResponse` object with the following structure:
```python theme={null}
class ModelResponse:
id: str
choices: List[Choices]
created: int
model: str
object: str
system_fingerprint: Optional[str]
usage: Usage
class Choices:
finish_reason: str
index: int
message: Message
class Message:
content: str
role: str
tool_calls: Optional[List[ChatCompletionMessageToolCall]]
class Usage:
prompt_tokens: int
completion_tokens: int
total_tokens: int
```
## Examples
### Basic Completion
```python theme={null}
from litellm import completion
response = completion(
model="gpt-4",
messages=[{"role": "user", "content": "Explain quantum computing in simple terms"}],
temperature=0.7,
max_tokens=200
)
print(response.choices[0].message.content)
print(f"Tokens used: {response.usage.total_tokens}")
```
### Using Different Providers
```python OpenAI theme={null}
response = completion(
model="gpt-4",
messages=[{"role": "user", "content": "Hello!"}],
api_key="sk-..."
)
```
```python Anthropic theme={null}
response = completion(
model="claude-3-5-sonnet-20241022",
messages=[{"role": "user", "content": "Hello!"}],
api_key="sk-ant-..."
)
```
```python Google theme={null}
response = completion(
model="gemini-pro",
messages=[{"role": "user", "content": "Hello!"}],
api_key="..."
)
```
```python Azure OpenAI theme={null}
response = completion(
model="azure/my-deployment",
messages=[{"role": "user", "content": "Hello!"}],
api_key="...",
api_base="https://my-resource.openai.azure.com",
api_version="2024-02-01"
)
```
### Structured Output with JSON
```python theme={null}
from litellm import completion
response = completion(
model="gpt-4",
messages=[{
"role": "user",
"content": "Extract the name and age: 'John is 30 years old'"
}],
response_format={"type": "json_object"}
)
print(response.choices[0].message.content) # {"name": "John", "age": 30}
```
### Structured Output with Pydantic
```python theme={null}
from litellm import completion
from pydantic import BaseModel
class Person(BaseModel):
name: str
age: int
response = completion(
model="gpt-4",
messages=[{
"role": "user",
"content": "Extract the name and age: 'John is 30 years old'"
}],
response_format=Person
)
person = Person.model_validate_json(response.choices[0].message.content)
print(person.name, person.age) # John 30
```
### System Messages and Context
```python theme={null}
response = completion(
model="gpt-4",
messages=[
{"role": "system", "content": "You are a helpful coding assistant. Be concise."},
{"role": "user", "content": "Write a Python function to reverse a string"},
],
temperature=0.3
)
```
### Setting Timeouts
```python theme={null}
import httpx
from litellm import completion
# Simple timeout
response = completion(
model="gpt-4",
messages=[{"role": "user", "content": "Hello"}],
timeout=30.0 # 30 seconds
)
# Advanced timeout with httpx.Timeout
response = completion(
model="gpt-4",
messages=[{"role": "user", "content": "Hello"}],
timeout=httpx.Timeout(connect=5.0, read=30.0, write=10.0, pool=5.0)
)
```
## Error Handling
```python theme={null}
from litellm import completion
from litellm.exceptions import (
AuthenticationError,
RateLimitError,
ContextWindowExceededError,
Timeout
)
try:
response = completion(
model="gpt-4",
messages=[{"role": "user", "content": "Hello"}]
)
except AuthenticationError as e:
print(f"Invalid API key: {e}")
except RateLimitError as e:
print(f"Rate limit exceeded: {e}")
except ContextWindowExceededError as e:
print(f"Context too large: {e}")
except Timeout as e:
print(f"Request timed out: {e}")
except Exception as e:
print(f"Unexpected error: {e}")
```
## Return Types
### Non-Streaming Response
Returns a `ModelResponse` object:
```python theme={null}
response = completion(
model="gpt-4",
messages=[{"role": "user", "content": "Hello"}]
)
print(response.id) # "chatcmpl-123"
print(response.model) # "gpt-4"
print(response.choices[0].message.content) # "Hello! How can I help you?"
print(response.usage.total_tokens) # 25
```
### Streaming Response
Returns a `CustomStreamWrapper` object. See [Streaming](/sdk/streaming) for details.
## Related
* [Streaming Responses](/sdk/streaming)
* [Async Operations](/sdk/async)
* [Function Calling](/sdk/function-calling)
* [Router for Load Balancing](/sdk/router)