> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/BerriAI/litellm/llms.txt
> Use this file to discover all available pages before exploring further.

# Deploy to Railway

> Deploy LiteLLM to Railway with automatic provisioning and GitHub integration

## Quick Deploy

Deploy LiteLLM to Railway with one click:

<a href="https://railway.app/template/HLP0Ub?referralCode=jch2ME" target="_blank">
  <img src="https://railway.app/button.svg" alt="Deploy on Railway" />
</a>

<Steps>
  <Step title="Click Deploy Button">
    Click the "Deploy on Railway" button above to open the template.
  </Step>

  <Step title="Configure Template">
    Railway will prompt you to set:

    * Project name
    * Environment variables (API keys)
    * Region preference
  </Step>

  <Step title="Deploy">
    Railway automatically:

    * Creates a new project
    * Provisions PostgreSQL database
    * Builds and deploys LiteLLM
    * Generates a public URL
  </Step>

  <Step title="Access Service">
    Once deployed, Railway provides:

    * **Public URL:** `https://litellm-production.up.railway.app`
    * **Custom domain:** Add your own domain in settings
  </Step>
</Steps>

## Manual Railway Deployment

### From GitHub Repository

<Steps>
  <Step title="Create New Project">
    1. Go to [Railway Dashboard](https://railway.app/dashboard)
    2. Click **New Project**
    3. Select **Deploy from GitHub repo**
    4. Choose `BerriAI/litellm` or your fork
  </Step>

  <Step title="Configure Service">
    Railway auto-detects Docker configuration from the repository.

    **Detected settings:**

    * **Runtime:** Docker
    * **Build:** Uses `/Dockerfile`
    * **Start command:** Automatic from ENTRYPOINT
  </Step>

  <Step title="Add PostgreSQL">
    1. Click **New Service** in your project
    2. Select **Database** → **PostgreSQL**
    3. Railway provisions database and injects `DATABASE_URL`
  </Step>

  <Step title="Set Environment Variables">
    Add required variables (see configuration section)
  </Step>

  <Step title="Deploy">
    Railway automatically builds and deploys on push to main branch.
  </Step>
</Steps>

### From Docker Image

<Steps>
  <Step title="Create New Project">
    1. Click **New Project**
    2. Select **Deploy a Docker Image**
  </Step>

  <Step title="Specify Image">
    ```bash theme={null}
    ghcr.io/berriai/litellm:main-stable
    ```
  </Step>

  <Step title="Configure Command">
    Add custom start command:

    ```bash theme={null}
    --port $PORT --config /app/config.yaml
    ```
  </Step>
</Steps>

## Database Setup

### Railway PostgreSQL

<Tip>
  Railway automatically sets `DATABASE_URL` when you add PostgreSQL to your project.
</Tip>

<Steps>
  <Step title="Add Database">
    1. In your project, click **New**
    2. Select **Database** → **PostgreSQL**
    3. Railway creates database with:
       * 1GB storage (Hobby plan)
       * Automatic backups
       * Private networking
  </Step>

  <Step title="Connection String">
    Railway automatically injects:

    ```bash theme={null}
    DATABASE_URL=postgresql://postgres:password@host:5432/railway
    ```

    No manual configuration needed!
  </Step>

  <Step title="Enable Model Storage">
    Add to your service:

    ```bash theme={null}
    STORE_MODEL_IN_DB=True
    ```
  </Step>
</Steps>

### Database Migrations

Migrations run automatically on container start using Prisma.

<Warning>
  First deployment takes 15-20 seconds while Prisma runs migrations and generates the client.
</Warning>

## Environment Configuration

### Required Variables

In Railway service settings, add:

```bash theme={null}
# Authentication
LITELLM_MASTER_KEY=sk-1234  # Generate a secure key

# Database (auto-injected if using Railway PostgreSQL)
# DATABASE_URL=postgresql://...  

# Enable UI model management
STORE_MODEL_IN_DB=True
```

### Provider API Keys

<Tabs>
  <Tab title="OpenAI">
    ```bash theme={null}
    OPENAI_API_KEY=sk-proj-...
    ```
  </Tab>

  <Tab title="Anthropic">
    ```bash theme={null}
    ANTHROPIC_API_KEY=sk-ant-...
    ```
  </Tab>

  <Tab title="Google Gemini">
    ```bash theme={null}
    GEMINI_API_KEY=AIza...
    ```
  </Tab>

  <Tab title="Azure OpenAI">
    ```bash theme={null}
    AZURE_API_KEY=your-key
    AZURE_API_BASE=https://your-resource.openai.azure.com
    AZURE_API_VERSION=2024-02-15-preview
    ```
  </Tab>
</Tabs>

### Optional Configuration

```bash theme={null}
# Redis for caching
REDIS_HOST=redis.railway.internal
REDIS_PORT=6379
REDIS_PASSWORD=your-password

# Observability
LANGFUSE_PUBLIC_KEY=pk-...
LANGFUSE_SECRET_KEY=sk-...

# Debug mode
LITELLM_LOG=DEBUG

# Custom port (Railway sets this automatically)
# PORT=4000
```

<Warning>
  Don't set `PORT` manually - Railway injects this automatically and maps to your service.
</Warning>

## Configuration with config.yaml

### Mount Configuration File

<Steps>
  <Step title="Create Config in Repo">
    Add `config.yaml` to your repository:

    ```yaml config.yaml theme={null}
    model_list:
      - model_name: gpt-4o
        litellm_params:
          model: gpt-4o
          api_key: os.environ/OPENAI_API_KEY
      
      - model_name: claude-sonnet-4
        litellm_params:
          model: anthropic/claude-sonnet-4-20250514
          api_key: os.environ/ANTHROPIC_API_KEY

    general_settings:
      master_key: os.environ/LITELLM_MASTER_KEY
      database_url: os.environ/DATABASE_URL
    ```
  </Step>

  <Step title="Update Start Command">
    In Railway service settings:

    ```bash theme={null}
    litellm --config /app/config.yaml --port $PORT
    ```
  </Step>

  <Step title="Redeploy">
    Push to GitHub or trigger manual deploy in Railway dashboard.
  </Step>
</Steps>

### Use Railway Variables

<Steps>
  <Step title="Create Variable Groups">
    Organize variables by environment:

    * **production** - Live environment
    * **staging** - Testing environment
    * **development** - Local development
  </Step>

  <Step title="Reference in Config">
    ```yaml theme={null}
    model_list:
      - model_name: gpt-4o
        litellm_params:
          model: gpt-4o
          api_key: os.environ/OPENAI_API_KEY
          api_base: os.environ/OPENAI_API_BASE  # Optional override

    general_settings:
      master_key: os.environ/LITELLM_MASTER_KEY
      database_url: os.environ/DATABASE_URL
    ```
  </Step>
</Steps>

## Custom Domain

<Steps>
  <Step title="Generate Railway Domain">
    Railway automatically provides:

    ```
    https://litellm-production-xxxx.up.railway.app
    ```
  </Step>

  <Step title="Add Custom Domain">
    1. In service settings, go to **Settings** → **Networking**
    2. Click **Add Custom Domain**
    3. Enter your domain: `api.yourdomain.com`
  </Step>

  <Step title="Configure DNS">
    Add a CNAME record:

    ```
    Type:  CNAME
    Name:  api
    Value: litellm-production-xxxx.up.railway.app
    TTL:   Auto
    ```
  </Step>

  <Step title="Wait for SSL">
    Railway automatically provisions SSL certificate (2-5 minutes).
  </Step>
</Steps>

<Tip>
  Railway provides automatic SSL for all domains using Let's Encrypt.
</Tip>

## Redis for Caching

### Add Redis Service

<Steps>
  <Step title="Add Redis to Project">
    1. Click **New** in your project
    2. Select **Database** → **Redis**
    3. Railway provisions Redis with:
       * 100MB storage (Hobby plan)
       * Private networking
       * Auto-injected connection details
  </Step>

  <Step title="Configure LiteLLM">
    Railway auto-injects Redis variables:

    ```bash theme={null}
    REDIS_URL=redis://default:password@redis.railway.internal:6379
    ```

    Or use individual variables:

    ```bash theme={null}
    REDIS_HOST=redis.railway.internal
    REDIS_PORT=6379
    REDIS_PASSWORD=auto-generated
    ```
  </Step>

  <Step title="Enable Caching">
    Update config.yaml:

    ```yaml theme={null}
    general_settings:
      cache: true
      redis_host: os.environ/REDIS_HOST
      redis_port: os.environ/REDIS_PORT
      redis_password: os.environ/REDIS_PASSWORD
    ```
  </Step>
</Steps>

## Private Networking

<Tip>
  Railway provides free private networking between services in the same project.
</Tip>

Services can communicate using:

```bash theme={null}
# Private DNS (recommended)
postgresql://user:pass@postgres.railway.internal:5432/db
redis://redis.railway.internal:6379

# Public URLs (avoid for internal communication)
https://service-name.up.railway.app
```

## Deployment Strategies

### Continuous Deployment

Railway automatically deploys on Git push:

```bash theme={null}
git add .
git commit -m "Update LiteLLM configuration"
git push origin main
```

Railway will:

1. Detect the push via webhook
2. Build Docker image
3. Run health checks
4. Deploy with zero downtime

### Manual Deployment

1. Go to service in Railway dashboard
2. Click **Deploy** → **Redeploy**
3. Or **Trigger Deploy** for specific commit

### Rollback

<Steps>
  <Step title="View Deployments">
    Click **Deployments** tab in service dashboard
  </Step>

  <Step title="Select Previous Version">
    Click on successful deployment
  </Step>

  <Step title="Rollback">
    Click **Redeploy this version**
  </Step>
</Steps>

## Monitoring and Observability

### Railway Observability

Built-in monitoring includes:

* **Logs:** Real-time log streaming
* **Metrics:** CPU, Memory, Network, Disk
* **Deployments:** Build and deploy history
* **Events:** Service lifecycle events

### View Logs

```bash theme={null}
# In Railway dashboard
1. Click on your service
2. Go to "Logs" tab
3. Filter by:
   - Time range
   - Log level (ERROR, INFO, DEBUG)
   - Search text
```

### Metrics Dashboard

Access in **Metrics** tab:

* **CPU usage:** % utilization over time
* **Memory usage:** MB used / available
* **Network:** Inbound/outbound traffic
* **Request count:** Requests per second

### External Monitoring

Add monitoring services:

<Tabs>
  <Tab title="Langfuse">
    ```bash theme={null}
    LANGFUSE_PUBLIC_KEY=pk-...
    LANGFUSE_SECRET_KEY=sk-...
    LANGFUSE_HOST=https://cloud.langfuse.com
    ```

    Add to config.yaml:

    ```yaml theme={null}
    litellm_settings:
      success_callback: ["langfuse"]
    ```
  </Tab>

  <Tab title="Datadog">
    ```bash theme={null}
    USE_DDTRACE=true
    DD_API_KEY=your-key
    DD_SITE=datadoghq.com
    DD_SERVICE=litellm-proxy
    DD_ENV=production
    ```
  </Tab>

  <Tab title="Sentry">
    ```bash theme={null}
    SENTRY_DSN=https://...@sentry.io/...
    SENTRY_ENVIRONMENT=production
    ```
  </Tab>
</Tabs>

## Scaling

### Vertical Scaling

Upgrade service resources:

1. Go to **Settings** → **Resources**
2. Select plan:
   * **Hobby:** 512MB RAM, 0.5 vCPU (\$5/month)
   * **Pro:** 8GB RAM, 8 vCPU (\$20/month)

### Horizontal Scaling

<Warning>
  Railway Hobby plan supports 1 instance only. Upgrade to Pro for multiple instances.
</Warning>

**Pro plan features:**

* Multiple service instances
* Load balancing
* Zero-downtime deployments
* Vertical and horizontal scaling

## Cost Management

### Pricing Overview

```yaml theme={null}
Hobby Plan ($5/month):
  - 512MB RAM, 0.5 vCPU per service
  - $5 free credit (covers small deployments)
  - 100GB bandwidth
  - Unlimited projects

Pro Plan (Usage-based):
  - $20/month base
  - Pay for resources used
  - 8GB RAM, 8 vCPU available
  - Priority support
```

### Cost Optimization Tips

<Steps>
  <Step title="Use Shared Database">
    Share PostgreSQL between staging/development environments.
  </Step>

  <Step title="Remove Unused Services">
    Delete stopped or unnecessary services to avoid charges.
  </Step>

  <Step title="Set Resource Limits">
    Configure memory and CPU limits to control costs:

    ```yaml theme={null}
    # In Railway service settings
    Memory Limit: 1GB
    CPU Limit: 1 vCPU
    ```
  </Step>

  <Step title="Use Sleep Mode">
    Enable auto-sleep for development environments (Hobby plan only).
  </Step>
</Steps>

## Troubleshooting

### Build Failures

```bash theme={null}
# Check build logs in Deployments tab

# Common issues:

# 1. Out of memory during build
Error: Docker build failed - OOM
Solution: Upgrade to Pro plan or optimize Dockerfile

# 2. npm/pip installation timeout
Error: Command timed out after 15 minutes
Solution: Use Docker layer caching, pre-build dependencies

# 3. File not found
Error: COPY failed: file not found: config.yaml
Solution: Ensure file is committed to Git
```

### Service Won't Start

```bash theme={null}
# Check logs for startup errors

# Missing DATABASE_URL
Error: DATABASE_URL not set
Solution: Add PostgreSQL service to project

# Port binding error
Error: Port 4000 already in use
Solution: Use $PORT variable, don't hardcode port

# Config file error
Error: Failed to load config.yaml
Solution: Validate YAML syntax, check file path
```

### Database Connection Issues

```bash theme={null}
# Use private networking
# ✅ Correct:
DATABASE_URL=postgresql://user:pass@postgres.railway.internal:5432/db

# ❌ Wrong (public URL, slower and less secure):
DATABASE_URL=postgresql://user:pass@xxx.railway.app:5432/db

# Test connection
railway run psql $DATABASE_URL
```

### High Memory Usage

```bash theme={null}
# Check metrics in Railway dashboard
# If memory usage > 80%:

# 1. Upgrade to larger instance
# 2. Enable Redis caching to reduce memory load
# 3. Optimize model configurations
# 4. Reduce concurrent requests
```

## Railway CLI

### Installation

```bash theme={null}
# Install Railway CLI
npm install -g @railway/cli

# Login
railway login

# Link to project
railway link
```

### Common Commands

```bash theme={null}
# View logs
railway logs

# Open service in browser
railway open

# Run command with Railway environment
railway run litellm --config config.yaml

# Deploy manually
railway up

# View environment variables
railway variables

# Set environment variable
railway variables set OPENAI_API_KEY=sk-...

# SSH into service
railway shell
```

## Comparison: Railway vs Render

| Feature               | Railway              | Render                  |
| --------------------- | -------------------- | ----------------------- |
| Free tier             | \$5 credit           | 90-day PostgreSQL trial |
| Pricing               | Usage-based          | Fixed tiers             |
| Auto-scaling          | Pro plan             | Yes                     |
| Private networking    | ✅ Free               | ❌                       |
| CLI                   | ✅                    | ❌                       |
| Build speed           | Fast (layer caching) | Medium                  |
| Zero-downtime deploys | ✅                    | ✅                       |

## Next Steps

<CardGroup cols={2}>
  <Card title="Fly.io Deployment" icon="plane" href="/deployment/fly-io">
    Deploy closer to users with edge compute
  </Card>

  <Card title="Monitoring" icon="chart-line" href="/deployment/monitoring">
    Set up observability and alerts
  </Card>

  <Card title="Security" icon="shield" href="/deployment/security">
    Secure your deployment
  </Card>

  <Card title="Performance" icon="gauge" href="/deployment/performance">
    Optimize for production workloads
  </Card>
</CardGroup>