# How to handle rate limits on LLM APIs

> Avoid Claude API rate limit errors by handling Retry-After headers, managing token throughput, and configuring daily spending limits.

**To avoid rate limit errors when calling Claude models, implement exponential backoff using the `Retry-After` header, throttle concurrent calls to your key tier, and raise daily spend or project budgets before reaching capacity.** The gateway exposes models like Claude Sonnet 5.5 (`vibiz/smart`), Claude Opus 5.5 (`vibiz/pro`), and Claude Haiku 5.5 (`vibiz/smart-mini`), as well as lab identifiers like `anthropic/claude-opus-5.5`. When a call is refused with an HTTP `429` status code, the response payload indicates whether the restriction is caused by request volume, token throughput, or daily financial caps.

Understanding the precise trigger for an error prevents your code from making redundant calls that deplete quotas. For an overview of common status codes across providers, see our guide on [LLM API errors explained: 429, 503 and context length](https://thelastapi.dev/blog/llm-api-errors-429-503).

## What triggers rate limit errors on Claude models?

Rate limits are enforced at two distinct levels: operational throughput (requests, tokens, and concurrency) and financial ceilings (daily account caps and project budgets). When you send a request to `/v1/ai/chat/completions`, the platform evaluates your usage against these parameters.

If your request exceeds any limit, the gateway answers with an HTTP `429` and a JSON error object containing `details.reason`.

| `details.reason` | Category | Default threshold | Primary resolution |
| --- | --- | --- | --- |
| `requests_per_minute` | Throughput | 120 (project key) / 300 (account key) | Pause using the duration in `Retry-After` |
| `tokens_per_minute` | Throughput | 200k (project key) / 400k (account key) | Lower `max_tokens` or queue inputs |
| `concurrency` | Throughput | 25 in flight (project) / 50 in flight (account); a call over it waits up to 60 s for a slot first | Limit active parallel asynchronous requests |
| `daily_spend` | Financial | $5 per account per UTC day | Raise safety cap in Billing or via CLI |
| `project_budget` | Financial | $1 per project per UTC day | Raise project daily budget in project settings |
| `end_user_budget` | Financial | App-defined via `end_user_daily_budget_usd` | Adjust individual end-user allocations |

Throughput limits evaluate short rolling windows, while financial limits reset at midnight UTC (`00:00:00Z`).

## How to inspect a 429 response

Every `429` response maintains an OpenAI-compatible error format. This allows standard client libraries to parse the failure cleanly:

```json
{
  "error": {
    "code": "rate_limited",
    "message": "Rate limit reached.",
    "type": "requests",
    "param": null,
    "details": {
      "reason": "requests_per_minute"
    }
  }
}
```

When receiving this payload, check two items immediately:
1. **The `Retry-After` response header:** When `details.reason` is `requests_per_minute`, `tokens_per_minute`, or `concurrency`, this header specifies the exact number of seconds to delay before repeating the call.
2. **The `details.reason` field:** If the reason is `daily_spend` or `project_budget`, retrying the call without raising the cap will fail continuously until midnight UTC.

If you stream responses using server-sent events (`stream: true`), an in-flight rate limit arrives as a final data block: `data: {"error": { ... }}` without a trailing `[DONE]`. For design patterns that help automated tools process these messages, read our guide to [agent-friendly APIs: errors that say what to do next](https://thelastapi.dev/blog/agent-friendly-api-errors).

## How to handle retries with backoff in code

Automated clients should read `Retry-After` headers and apply jitter when scheduling retries. This prevents multiple concurrent workers from synchronizing their retry attempts.

Here is an example using the OpenAI SDK connected to the unified gateway via environment variables defined in [AI docs](https://thelastapi.dev/docs/ai):

```ts
import OpenAI from 'openai';

const openai = new OpenAI({
  baseURL: process.env.VIBIZ_AI_BASE_URL,
  apiKey: process.env.VIBIZ_AI_API_KEY,
});

async function callClaudeWithRetry(prompt: string, maxRetries = 3) {
  for (let attempt = 0; attempt <= maxRetries; attempt++) {
    try {
      return await openai.chat.completions.create({
        model: 'vibiz/smart',
        messages: [{ role: 'user', content: prompt }],
        max_tokens: 1024,
      });
    } catch (error: any) {
      if (error?.status === 429) {
        const reason = error?.error?.details?.reason;

        // Permanent failures for the current UTC day
        if (reason === 'daily_spend' || reason === 'project_budget') {
          throw new Error(`Execution halted: ${reason} limit reached. Raise budgets to continue.`);
        }

        if (attempt === maxRetries) {
          throw error;
        }

        // Extract Retry-After header or default to backoff with jitter
        const retryHeader = error?.headers?.['retry-after'];
        const delayMs = retryHeader
          ? parseInt(retryHeader, 10) * 1000
          : Math.pow(2, attempt) * 1000 + Math.random() * 500;

        await new Promise((resolve) => setTimeout(resolve, delayMs));
        continue;
      }
      throw error;
    }
  }
}
```

This pattern stops retries when budgets are depleted, avoiding unnecessary API consumption while resolving short-term throughput throttles. Learn more about unified configurations in our guide on [one API key for GPT, Claude and Gemini](https://thelastapi.dev/blog/one-api-key-gpt-claude-gemini).

## How to raise project and account spending limits

When `details.reason` returns `daily_spend` or `project_budget`, calls cannot proceed until an owner increases the threshold or limits reset at midnight UTC. The platform sends automated warning emails at 80% and 100% of these limits, containing links directly to the setting that needs adjustment.

You can view current allocations from the CLI:

```bash
vibiz usage
```

To modify these caps using the CLI or HTTP API, use a full-access key as outlined in [Keys and environments](https://thelastapi.dev/docs/keys):

### Adjusting account daily safety caps
The account safety cap defaults to $5 per day to prevent runaway processes. Change it up to $1,000 using the CLI:

```bash
vibiz billing limit 50
```

Or make an HTTP request to the billing endpoint:

```bash
curl -X PATCH https://thelastapi.dev/v1/billing \
  -H "Authorization: Bearer $VIBIZ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"daily_spend_limit_usd": 50}'
```

### Adjusting project AI budgets
Projects default to a $1 daily AI budget. Inspect and adjust this limit using:

```bash
vibiz ai budget 10 --project prj_123
```

Or update the project configuration over HTTP:

```bash
curl -X PATCH https://thelastapi.dev/v1/projects/prj_123/ai \
  -H "Authorization: Bearer $VIBIZ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"daily_budget_usd": 10}'
```

Setting `daily_budget_usd` to `null` removes the project budget restriction entirely, leaving only the account-level ceiling in effect. Review the [Usage and limits](https://thelastapi.dev/docs/usage) documentation for details on limit notification settings.

## How to structure workloads to stay under Claude throughput caps

To eliminate `requests_per_minute` and `tokens_per_minute` errors without relying solely on retry loops, manage your application's concurrency and token reservations:

1. **Choose the appropriate key type:** A project key (`vz_ai_...`) defaults to 120 requests and 200,000 tokens a minute with 25 concurrent requests in flight. It can be raised up to 600 requests, 2,000,000 tokens a minute, and 50 concurrent requests. Account keys (`vz_sk_...`) provide 300 requests and 400,000 tokens a minute with 50 concurrent requests.
2. **Explicitly pass `max_tokens`:** When omitted, `max_tokens` defaults to 1,024 (up to 16,384 per call). Before dispatching each request, the gateway reserves balance based on its worst-case calculation. Specifying a realistic `max_tokens` ensures you do not trigger pre-execution balance failures or tie up available concurrency.
3. **Account for long prompts:** Claude Haiku 5.5 (`vibiz/smart-mini`) applies higher tier pricing above 100,000 input tokens. When submitting large context windows, batch prompts sequentially rather than issuing multiple parallel requests against the token-per-minute cap.
4. **Throttle client-side queues:** Restrict your application worker pools to stay below your key's concurrency limit (25 for standard project keys, 50 for account keys; a call over it waits up to 60 s for a slot before a 429).

To monitor live token volumes across tools, models, and end users, run:

```bash
vibiz usage --period 24h --by model
```

To configure key quotas, raise spending thresholds, or set up automated credit replenishment, review our guides on [AI](https://thelastapi.dev/docs/ai) and [Billing and credits](https://thelastapi.dev/docs/billing).

## Frequently asked questions

### What is the difference between a project key and an account key rate limit?

An account key allows 300 requests and 400,000 tokens per minute with 50 in-flight calls. A project key defaults to 120 requests and 200,000 tokens per minute with 25 in-flight calls, but an owner can raise its limits up to 600 requests, 2,000,000 tokens, and 50 in-flight calls.

### Why do I get a 429 rate limit error when my request volume is low?

An HTTP 429 status code is returned for both throughput limits and spending caps. Inspect `error.details.reason`: values such as `daily_spend` (account ceiling) or `project_budget` (project ceiling) indicate a financial limit has been reached rather than a per-minute request spike.

### At what time do daily spend limits reset?

All daily spend ceilings and project AI budgets reset at midnight UTC (00:00:00Z). The reset timestamp is also provided in usage payloads under `limits.resets_at`.

### Can Claude model calls switch to another model automatically during rate limits?

No. thelastapi never substitutes another model when your requested model encounters rate limits or availability issues. The gateway returns an explicit error code so your application maintains predictable output and cost.

---

Published 2026-11-15 · https://thelastapi.dev/blog/handle-llm-rate-limits
