> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/BerriAI/litellm/llms.txt
> Use this file to discover all available pages before exploring further.

# Retry Mechanisms

> Automatic retry logic for failed requests with exponential backoff

## Overview

LiteLLM automatically retries failed requests with intelligent backoff strategies. Retries help handle transient failures like rate limits, timeouts, and temporary service disruptions.

## Default Retry Behavior

By default, LiteLLM retries requests **2 times** (3 total attempts including the initial request).

```python theme={null}
import litellm
from litellm import completion

# Uses default retry behavior (2 retries)
response = completion(
    model="gpt-4",
    messages=[{"role": "user", "content": "Hello!"}]
)
```

## Configuring Retries

### Global Retry Configuration

Set retries for all requests:

```python theme={null}
import litellm

# Set global retry count
litellm.num_retries = 3

response = completion(
    model="gpt-4",
    messages=[{"role": "user", "content": "Hello!"}]
)
# Will retry up to 3 times on failure
```

### Per-Request Retry Configuration

```python theme={null}
response = completion(
    model="gpt-4",
    messages=[{"role": "user", "content": "Hello!"}],
    num_retries=5  # Override global setting
)
```

### Router Retry Configuration

```python theme={null}
from litellm import Router

router = Router(
    model_list=[
        {
            "model_name": "gpt-4",
            "litellm_params": {
                "model": "gpt-4",
                "api_key": "sk-..."
            }
        }
    ],
    num_retries=3,  # Default retries for all models
    retry_after=5  # Wait 5 seconds before first retry
)
```

## Retry Policies

Customize retry behavior based on error type:

### Basic Retry Policy

```python theme={null}
from litellm import Router
from litellm.types.router import RetryPolicy

router = Router(
    model_list=[...],
    retry_policy=RetryPolicy(
        TimeoutErrorRetries=5,
        RateLimitErrorRetries=3,
        InternalServerErrorRetries=2,
        ContentPolicyViolationErrorRetries=0,  # Don't retry these
        AuthenticationErrorRetries=0  # Don't retry auth errors
    )
)
```

### Available Error Types

Configure retries for specific error types:

* `TimeoutErrorRetries` - Connection/request timeouts
* `RateLimitErrorRetries` - Rate limit (429) errors
* `InternalServerErrorRetries` - Server errors (500, 502, 503, 504)
* `BadRequestErrorRetries` - Bad request (400) errors
* `AuthenticationErrorRetries` - Authentication (401, 403) errors
* `ContentPolicyViolationErrorRetries` - Content filtering errors
* `UnsupportedParamsRetries` - Unsupported parameter errors

### Model Group Retry Policies

Set different retry policies for different model groups:

```python theme={null}
from litellm import Router
from litellm.types.router import RetryPolicy

router = Router(
    model_list=[
        {"model_name": "gpt-4", "litellm_params": {...}},
        {"model_name": "gpt-3.5", "litellm_params": {...}},
        {"model_name": "claude", "litellm_params": {...}}
    ],
    # Global retry policy
    retry_policy=RetryPolicy(
        TimeoutErrorRetries=3,
        RateLimitErrorRetries=2
    ),
    # Model-specific retry policies
    model_group_retry_policy={
        "gpt-4": RetryPolicy(
            TimeoutErrorRetries=5,  # More retries for expensive model
            RateLimitErrorRetries=10
        ),
        "claude": RetryPolicy(
            TimeoutErrorRetries=2,
            RateLimitErrorRetries=1
        )
    }
)
```

## Retry Timing

### Retry After

Set minimum wait time before retrying:

```python theme={null}
router = Router(
    model_list=[...],
    retry_after=10  # Wait at least 10 seconds before retry
)
```

### Exponential Backoff

LiteLLM uses exponential backoff automatically:

```python theme={null}
# Retry timing with exponential backoff:
# 1st retry: 0-2 seconds
# 2nd retry: 0-4 seconds  
# 3rd retry: 0-8 seconds
# 4th retry: 0-16 seconds
# etc.

response = completion(
    model="gpt-4",
    messages=[{"role": "user", "content": "Hello!"}],
    num_retries=4
)
```

<Info>
  Exponential backoff helps avoid overwhelming rate-limited services and increases the chance of successful retries.
</Info>

## Streaming with Retries

Retries work with streaming responses:

```python theme={null}
response = completion(
    model="gpt-4",
    messages=[{"role": "user", "content": "Write a story"}],
    stream=True,
    num_retries=3
)

for chunk in response:
    print(chunk.choices[0].delta.content, end="")
```

## Async Retries

Retries work seamlessly with async operations:

```python theme={null}
import asyncio
from litellm import acompletion

async def make_request():
    response = await acompletion(
        model="gpt-4",
        messages=[{"role": "user", "content": "Hello!"}],
        num_retries=5
    )
    return response

response = asyncio.run(make_request())
```

## Monitoring Retries

### Custom Retry Logging

```python theme={null}
from litellm.integrations import CustomLogger
import litellm

class RetryLogger(CustomLogger):
    def log_failure_event(self, kwargs, response_obj, start_time, end_time):
        print(f"Request failed: {kwargs.get('model')}")
        print(f"Exception: {kwargs.get('exception')}")
        print(f"Retry attempt: {kwargs.get('litellm_call_id')}")
    
    def log_success_event(self, kwargs, response_obj, start_time, end_time):
        print(f"Request succeeded after retries")

litellm.callbacks = [RetryLogger()]
```

### Response Headers

Retry information is included in response metadata:

```python theme={null}
response = router.completion(
    model="gpt-4",
    messages=[{"role": "user", "content": "Hello"}],
    num_retries=3
)

# Check retry headers
retry_count = response._hidden_params.get("retry_count", 0)
print(f"Number of retries: {retry_count}")
```

## Retry vs Fallback

<Card title="Understanding the Difference">
  **Retries**: Attempt the same model/deployment multiple times

  **Fallbacks**: Switch to a different model/deployment after retries fail

  **Execution Order**: LiteLLM tries retries first, then fallbacks
</Card>

### Combined Retry and Fallback

```python theme={null}
router = Router(
    model_list=[
        {"model_name": "gpt-4", "litellm_params": {...}},
        {"model_name": "gpt-3.5", "litellm_params": {...}}
    ],
    num_retries=3,  # Retry each model 3 times
    fallbacks=[{"gpt-4": ["gpt-3.5"]}]  # Then fallback to gpt-3.5
)

# Execution flow:
# 1. Try gpt-4 (retry up to 3 times)
# 2. If all retries fail, try gpt-3.5 (retry up to 3 times)
```

## Best Practices

1. **Set appropriate retry counts**
   * More retries for critical requests
   * Fewer retries for latency-sensitive applications

2. **Configure by error type**
   * Retry timeouts and rate limits aggressively
   * Don't retry authentication or validation errors

3. **Use exponential backoff**
   * Already built-in, respects API rate limits

4. **Monitor retry rates**
   * High retry rates indicate underlying issues
   * Track which models/deployments need retries

5. **Combine with fallbacks**
   * Use retries for transient failures
   * Use fallbacks for persistent failures

6. **Set timeouts**
   * Prevent retries from taking too long
   ```python theme={null}
   response = completion(
       model="gpt-4",
       messages=[{"role": "user", "content": "Hello"}],
       num_retries=3,
       timeout=30  # Total timeout for all retries
   )
   ```

## Common Retry Scenarios

### Rate Limit Handling

```python theme={null}
from litellm import Router
from litellm.types.router import RetryPolicy

router = Router(
    model_list=[...],
    retry_policy=RetryPolicy(
        RateLimitErrorRetries=10  # Retry rate limits many times
    ),
    retry_after=60  # Wait 1 minute before first retry
)
```

### Timeout Handling

```python theme={null}
router = Router(
    model_list=[...],
    retry_policy=RetryPolicy(
        TimeoutErrorRetries=5
    ),
    timeout=30,  # 30 second timeout per request
    stream_timeout=60  # 60 second timeout for streaming
)
```

### Production Configuration

```python theme={null}
router = Router(
    model_list=[...],
    retry_policy=RetryPolicy(
        TimeoutErrorRetries=3,
        RateLimitErrorRetries=5,
        InternalServerErrorRetries=2,
        ContentPolicyViolationErrorRetries=0,
        AuthenticationErrorRetries=0
    ),
    num_retries=3,
    retry_after=5,
    timeout=60
)
```

## Disabling Retries

```python theme={null}
# Disable retries globally
litellm.num_retries = 0

# Or per request
response = completion(
    model="gpt-4",
    messages=[{"role": "user", "content": "Hello!"}],
    num_retries=0
)
```

## Error Handling

```python theme={null}
from litellm import completion
from litellm.exceptions import (
    Timeout,
    RateLimitError,
    ServiceUnavailableError
)

try:
    response = completion(
        model="gpt-4",
        messages=[{"role": "user", "content": "Hello!"}],
        num_retries=3
    )
except RateLimitError as e:
    print(f"Rate limited after retries: {str(e)}")
except Timeout as e:
    print(f"Timeout after retries: {str(e)}")
except Exception as e:
    print(f"Failed after all retries: {str(e)}")
```

## Related Features

* [Fallbacks](/features/fallbacks) - Switch to different models after retries
* [Load Balancing](/features/loadbalancing) - Distribute across multiple deployments
* [Cost Tracking](/features/cost-tracking) - Monitor costs of retries
* [Logging](/features/logging) - Track retry attempts