> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/BerriAI/litellm/llms.txt
> Use this file to discover all available pages before exploring further.

# Budget Alerts & Monitoring

> Configure budget tracking, alerts, and spending notifications

## Overview

LiteLLM Proxy provides comprehensive budget tracking and alerting capabilities:

* **Budget Enforcement**: Hard limits to prevent overspending
* **Soft Budgets**: Warnings before hitting limits
* **Webhook Alerts**: Real-time notifications
* **Slack Integration**: Team notifications
* **Spend Tracking**: Monitor usage across keys, users, and teams

## Budget Types

### Global Budget

Set a budget for the entire proxy:

```yaml config.yaml theme={null}
litellm_settings:
  max_budget: 1000.0       # Maximum spend in USD
  budget_duration: 30d     # Budget reset period
```

When the global budget is exceeded, all requests are rejected:

```json theme={null}
{
  "error": {
    "message": "Proxy budget exceeded. Max budget: $1000.00",
    "type": "budget_exceeded"
  }
}
```

### Key Budget

Set budgets per virtual key:

```bash theme={null}
curl -X POST 'http://localhost:4000/key/generate' \
  -H 'Authorization: Bearer sk-1234' \
  -H 'Content-Type: application/json' \
  -d '{
    "models": ["gpt-3.5-turbo"],
    "max_budget": 100.0,
    "budget_duration": "30d"
  }'
```

### Team Budget

Set budgets for teams:

```bash theme={null}
curl -X POST 'http://localhost:4000/team/new' \
  -H 'Authorization: Bearer sk-1234' \
  -H 'Content-Type: application/json' \
  -d '{
    "team_alias": "engineering",
    "max_budget": 1000.0,
    "budget_duration": "30d"
  }'
```

### User Budget

Set budgets per user:

```bash theme={null}
curl -X POST 'http://localhost:4000/user/new' \
  -H 'Authorization: Bearer sk-1234' \
  -H 'Content-Type: application/json' \
  -d '{
    "user_id": "john@example.com",
    "max_budget": 50.0,
    "budget_duration": "30d"
  }'
```

## Soft Budgets (Alerts)

Soft budgets trigger alerts without blocking requests:

```bash theme={null}
curl -X POST 'http://localhost:4000/key/generate' \
  -H 'Authorization: Bearer sk-1234' \
  -H 'Content-Type: application/json' \
  -d '{
    "models": ["gpt-3.5-turbo"],
    "max_budget": 100.0,
    "soft_budget": 80.0,     # Alert at $80
    "budget_duration": "30d"
  }'
```

<Steps>
  <Step title="Set Soft Budget">
    Configure soft budget threshold (80% of max budget)
  </Step>

  <Step title="Monitor Usage">
    Proxy tracks spending in real-time
  </Step>

  <Step title="Receive Alert">
    Webhook/Slack notification when soft budget is exceeded
  </Step>

  <Step title="Take Action">
    Increase budget or optimize usage before hitting hard limit
  </Step>
</Steps>

## Slack Alerts

Configure Slack notifications for budget alerts:

### Setup

<Steps>
  <Step title="Create Slack Webhook">
    1. Go to [https://api.slack.com/apps](https://api.slack.com/apps)
    2. Create a new app
    3. Enable Incoming Webhooks
    4. Create a webhook URL
  </Step>

  <Step title="Configure Proxy">
    Add Slack settings to your config:

    ```yaml config.yaml theme={null}
    litellm_settings:
      alerting: ["slack"]
      alerting_threshold: 0.8  # Alert at 80% budget
      
      # Slack webhook URL
      slack_webhook_url: os.environ/SLACK_WEBHOOK_URL
      
      # Optional: Customize alert message
      slack_alert_to_webhook_url: os.environ/SLACK_WEBHOOK_URL
    ```
  </Step>

  <Step title="Set Environment Variable">
    ```bash theme={null}
    export SLACK_WEBHOOK_URL="https://hooks.slack.com/services/..."
    ```
  </Step>

  <Step title="Restart Proxy">
    ```bash theme={null}
    litellm --config config.yaml
    ```
  </Step>
</Steps>

### Alert Example

When a budget threshold is reached, Slack receives:

```
🚨 Budget Alert

Key: sk-1234...abcd
Spend: $82.45 / $100.00 (82%)
Budget Reset: 2024-04-15 10:30:00 UTC

Consider increasing budget or optimizing usage.
```

## Webhook Alerts

Send budget alerts to custom webhooks:

```yaml config.yaml theme={null}
litellm_settings:
  alerting: ["webhook"]
  alerting_threshold: 0.8
  
  # Custom webhook URL
  webhook_url: https://example.com/alerts
```

Webhook payload:

```json theme={null}
{
  "event": "budget_alert",
  "timestamp": "2024-04-15T10:30:00Z",
  "key": "sk-1234...abcd",
  "key_alias": "production-key",
  "spend": 82.45,
  "max_budget": 100.0,
  "budget_threshold": 80.0,
  "percentage": 0.8245,
  "budget_reset_at": "2024-05-15T10:30:00Z",
  "metadata": {
    "environment": "production",
    "team": "backend"
  }
}
```

## Email Alerts

Configure email notifications:

```yaml config.yaml theme={null}
litellm_settings:
  alerting: ["email"]
  alerting_threshold: 0.8
  
  # Email settings
  email_alerts_to: ["admin@example.com", "finance@example.com"]
  
  # SMTP configuration
  smtp_host: os.environ/SMTP_HOST
  smtp_port: 587
  smtp_user: os.environ/SMTP_USER
  smtp_password: os.environ/SMTP_PASSWORD
  smtp_sender: alerts@example.com
```

Required environment variables:

```bash theme={null}
export SMTP_HOST="smtp.gmail.com"
export SMTP_USER="your-email@gmail.com"
export SMTP_PASSWORD="your-app-password"
```

## Budget Reset Schedule

Budgets automatically reset based on the configured duration:

<CodeGroup>
  ```yaml Daily Reset theme={null}
  max_budget: 100.0
  budget_duration: 1d
  ```

  ```yaml Weekly Reset theme={null}
  max_budget: 500.0
  budget_duration: 7d
  ```

  ```yaml Monthly Reset theme={null}
  max_budget: 1000.0
  budget_duration: 30d
  ```

  ```yaml No Auto-Reset theme={null}
  max_budget: 1000.0
  budget_duration: null
  ```
</CodeGroup>

### Manual Budget Reset

Reset a key's budget manually:

```bash theme={null}
curl -X POST 'http://localhost:4000/key/update' \
  -H 'Authorization: Bearer sk-1234' \
  -H 'Content-Type: application/json' \
  -d '{
    "key": "sk-1234567890abcdef",
    "spend": 0.0  # Reset spend to 0
  }'
```

## Spend Tracking

### View Key Spend

```bash theme={null}
curl -X GET 'http://localhost:4000/key/info' \
  -H 'Authorization: Bearer sk-1234567890abcdef'
```

Response:

```json theme={null}
{
  "key": "sk-1234...def",
  "spend": 45.23,
  "max_budget": 100.0,
  "budget_reset_at": "2024-04-15T10:30:00Z",
  "models": ["gpt-3.5-turbo"]
}
```

### View All Keys Spend

```bash theme={null}
curl -X GET 'http://localhost:4000/spend/keys' \
  -H 'Authorization: Bearer sk-1234'
```

### View Team Spend

```bash theme={null}
curl -X GET 'http://localhost:4000/spend/teams' \
  -H 'Authorization: Bearer sk-1234'
```

### View User Spend

```bash theme={null}
curl -X GET 'http://localhost:4000/spend/users' \
  -H 'Authorization: Bearer sk-1234'
```

## Projected Spend Alerts

Alert based on projected monthly spend:

```yaml config.yaml theme={null}
general_settings:
  projected_spend_alerts: true
  projected_spend_threshold: 0.9  # Alert at 90% of projected

litellm_settings:
  alerting: ["slack"]
  slack_webhook_url: os.environ/SLACK_WEBHOOK_URL
```

If current usage trends suggest exceeding budget, alerts are sent proactively.

## Advanced Alerting

### Multi-Channel Alerts

Send to multiple channels:

```yaml config.yaml theme={null}
litellm_settings:
  alerting: ["slack", "webhook", "email"]
  alerting_threshold: 0.8
  
  slack_webhook_url: os.environ/SLACK_WEBHOOK_URL
  webhook_url: https://example.com/alerts
  email_alerts_to: ["admin@example.com"]
```

### Team-Specific Alerts

Configure different alert settings per team:

```yaml config.yaml theme={null}
litellm_settings:
  default_team_settings:
    - team_id: team-engineering
      alerting: ["slack"]
      alerting_threshold: 0.8
      slack_webhook_url: os.environ/ENGINEERING_SLACK_WEBHOOK
    
    - team_id: team-marketing
      alerting: ["email"]
      alerting_threshold: 0.9
      email_alerts_to: ["marketing@example.com"]
```

### Custom Alert Thresholds

```yaml config.yaml theme={null}
litellm_settings:
  # Alert at multiple thresholds
  alerting_thresholds: [0.5, 0.75, 0.9, 1.0]
  
  alerting: ["slack"]
  slack_webhook_url: os.environ/SLACK_WEBHOOK_URL
```

Alerts are sent at 50%, 75%, 90%, and 100% of budget.

## Monitoring Dashboard

### Admin UI

Access the admin dashboard at `http://localhost:4000/ui`:

* **Overview**: Total spend across all keys
* **Keys**: Individual key spend and budgets
* **Teams**: Team-level spend tracking
* **Users**: Per-user spend analysis
* **Charts**: Spend trends over time

### Prometheus Metrics

Export budget metrics to Prometheus:

```yaml config.yaml theme={null}
litellm_settings:
  success_callback: ["prometheus"]
```

Metrics available:

* `litellm_spend_total` - Total spend
* `litellm_key_spend` - Per-key spend
* `litellm_team_spend` - Per-team spend
* `litellm_budget_remaining` - Remaining budget

Query in Prometheus:

```promql theme={null}
# Keys approaching budget limit
litellm_key_spend / litellm_key_budget > 0.8

# Total proxy spend
sum(litellm_spend_total)

# Spend by model
sum by (model) (litellm_spend_total)
```

## Best Practices

### 1. Set Conservative Budgets

Start with lower budgets and increase as needed:

```json theme={null}
{
  "max_budget": 10.0,      // Start small
  "soft_budget": 8.0,      // Early warning
  "duration": "7d"         // Short period
}
```

### 2. Use Soft Budgets

Always set soft budgets for early warnings:

```json theme={null}
{
  "max_budget": 100.0,
  "soft_budget": 80.0     // 80% threshold
}
```

### 3. Monitor Regularly

Check spend daily or weekly:

```bash theme={null}
# Daily spend check
curl -X GET 'http://localhost:4000/spend/keys' \
  -H 'Authorization: Bearer sk-1234' \
  | jq '.[] | select(.spend > .max_budget * 0.8)'
```

### 4. Team Budgets

Use team budgets for organizational cost allocation:

```bash theme={null}
# Engineering team
curl -X POST 'http://localhost:4000/team/new' \
  -d '{
    "team_alias": "engineering",
    "max_budget": 5000.0,
    "budget_duration": "30d"
  }'

# Marketing team
curl -X POST 'http://localhost:4000/team/new' \
  -d '{
    "team_alias": "marketing",
    "max_budget": 1000.0,
    "budget_duration": "30d"
  }'
```

### 5. Metadata for Cost Tracking

Use metadata to track costs by project:

```json theme={null}
{
  "metadata": {
    "project": "chatbot-v2",
    "cost_center": "engineering",
    "environment": "production"
  }
}
```

## Troubleshooting

### Alerts Not Firing

Check configuration:

```bash theme={null}
# Verify alerting is enabled
curl -X GET 'http://localhost:4000/config' \
  -H 'Authorization: Bearer sk-1234' \
  | jq '.litellm_settings.alerting'

# Test Slack webhook
curl -X POST $SLACK_WEBHOOK_URL \
  -H 'Content-Type: application/json' \
  -d '{"text": "Test alert"}'
```

### Incorrect Spend Tracking

Verify database is configured:

```yaml config.yaml theme={null}
general_settings:
  database_url: os.environ/DATABASE_URL
  store_model_in_db: true
```

### Budget Not Resetting

Check budget reset schedule:

```bash theme={null}
curl -X GET 'http://localhost:4000/key/info' \
  -H 'Authorization: Bearer sk-1234567890abcdef' \
  | jq '.budget_reset_at'
```

## Next Steps

<CardGroup cols={2}>
  <Card title="Virtual Keys" icon="key" href="/proxy/virtual-keys">
    Learn about key management
  </Card>

  <Card title="Configuration" icon="gear" href="/proxy/configs">
    Advanced configuration options
  </Card>

  <Card title="Docker Deployment" icon="docker" href="/proxy/docker-deployment">
    Deploy in production
  </Card>

  <Card title="Quick Start" icon="rocket" href="/proxy/quick-start">
    Get started guide
  </Card>
</CardGroup>