How to handle rate limits on LLM APIs
Avoid Claude API rate limit errors by handling Retry-After headers, managing token throughput, and configuring daily spending limits.
On this page 6 sections
To avoid rate limit errors when calling Claude models, implement exponential backoff using the Retry-After header, throttle concurrent calls to your key tier, and raise daily spend or project budgets before reaching capacity. The gateway exposes models like Claude Sonnet 5.5 (vibiz/smart), Claude Opus 5.5 (vibiz/pro), and Claude Haiku 5.5 (vibiz/smart-mini), as well as lab identifiers like anthropic/claude-opus-5.5. When a call is refused with an HTTP 429 status code, the response payload indicates whether the restriction is caused by request volume, token throughput, or daily financial caps.
Understanding the precise trigger for an error prevents your code from making redundant calls that deplete quotas. For an overview of common status codes across providers, see our guide on LLM API errors explained: 429, 503 and context length.
What triggers rate limit errors on Claude models?
Rate limits are enforced at two distinct levels: operational throughput (requests, tokens, and concurrency) and financial ceilings (daily account caps and project budgets). When you send a request to /v1/ai/chat/completions, the platform evaluates your usage against these parameters.
If your request exceeds any limit, the gateway answers with an HTTP 429 and a JSON error object containing details.reason.
details.reason | Category | Default threshold | Primary resolution |
|---|---|---|---|
requests_per_minute | Throughput | 120 (project key) / 300 (account key) | Pause using the duration in Retry-After |
tokens_per_minute | Throughput | 200k (project key) / 400k (account key) | Lower max_tokens or queue inputs |
concurrency | Throughput | 25 in flight (project) / 50 in flight (account); a call over it waits up to 60 s for a slot first | Limit active parallel asynchronous requests |
daily_spend | Financial | $5 per account per UTC day | Raise safety cap in Billing or via CLI |
project_budget | Financial | $1 per project per UTC day | Raise project daily budget in project settings |
end_user_budget | Financial | App-defined via end_user_daily_budget_usd | Adjust individual end-user allocations |
Throughput limits evaluate short rolling windows, while financial limits reset at midnight UTC (00:00:00Z).
How to inspect a 429 response
Every 429 response maintains an OpenAI-compatible error format. This allows standard client libraries to parse the failure cleanly:
{
"error": {
"code": "rate_limited",
"message": "Rate limit reached.",
"type": "requests",
"param": null,
"details": {
"reason": "requests_per_minute"
}
}
}When receiving this payload, check two items immediately:
- The
Retry-Afterresponse header: Whendetails.reasonisrequests_per_minute,tokens_per_minute, orconcurrency, this header specifies the exact number of seconds to delay before repeating the call. - The
details.reasonfield: If the reason isdaily_spendorproject_budget, retrying the call without raising the cap will fail continuously until midnight UTC.
If you stream responses using server-sent events (stream: true), an in-flight rate limit arrives as a final data block: data: {"error": { ... }} without a trailing [DONE]. For design patterns that help automated tools process these messages, read our guide to agent-friendly APIs: errors that say what to do next.
How to handle retries with backoff in code
Automated clients should read Retry-After headers and apply jitter when scheduling retries. This prevents multiple concurrent workers from synchronizing their retry attempts.
Here is an example using the OpenAI SDK connected to the unified gateway via environment variables defined in AI docs:
import OpenAI from 'openai';
const openai = new OpenAI({
baseURL: process.env.VIBIZ_AI_BASE_URL,
apiKey: process.env.VIBIZ_AI_API_KEY,
});
async function callClaudeWithRetry(prompt: string, maxRetries = 3) {
for (let attempt = 0; attempt <= maxRetries; attempt++) {
try {
return await openai.chat.completions.create({
model: 'vibiz/smart',
messages: [{ role: 'user', content: prompt }],
max_tokens: 1024,
});
} catch (error: any) {
if (error?.status === 429) {
const reason = error?.error?.details?.reason;
// Permanent failures for the current UTC day
if (reason === 'daily_spend' || reason === 'project_budget') {
throw new Error(`Execution halted: ${reason} limit reached. Raise budgets to continue.`);
}
if (attempt === maxRetries) {
throw error;
}
// Extract Retry-After header or default to backoff with jitter
const retryHeader = error?.headers?.['retry-after'];
const delayMs = retryHeader
? parseInt(retryHeader, 10) * 1000
: Math.pow(2, attempt) * 1000 + Math.random() * 500;
await new Promise((resolve) => setTimeout(resolve, delayMs));
continue;
}
throw error;
}
}
}This pattern stops retries when budgets are depleted, avoiding unnecessary API consumption while resolving short-term throughput throttles. Learn more about unified configurations in our guide on one API key for GPT, Claude and Gemini.
How to raise project and account spending limits
When details.reason returns daily_spend or project_budget, calls cannot proceed until an owner increases the threshold or limits reset at midnight UTC. The platform sends automated warning emails at 80% and 100% of these limits, containing links directly to the setting that needs adjustment.
You can view current allocations from the CLI:
vibiz usageTo modify these caps using the CLI or HTTP API, use a full-access key as outlined in Keys and environments:
Adjusting account daily safety caps
The account safety cap defaults to $5 per day to prevent runaway processes. Change it up to $1,000 using the CLI:
vibiz billing limit 50Or make an HTTP request to the billing endpoint:
curl -X PATCH https://thelastapi.dev/v1/billing \
-H "Authorization: Bearer $VIBIZ_API_KEY" \
-H "Content-Type: application/json" \
-d '{"daily_spend_limit_usd": 50}'Adjusting project AI budgets
Projects default to a $1 daily AI budget. Inspect and adjust this limit using:
vibiz ai budget 10 --project prj_123Or update the project configuration over HTTP:
curl -X PATCH https://thelastapi.dev/v1/projects/prj_123/ai \
-H "Authorization: Bearer $VIBIZ_API_KEY" \
-H "Content-Type: application/json" \
-d '{"daily_budget_usd": 10}'Setting daily_budget_usd to null removes the project budget restriction entirely, leaving only the account-level ceiling in effect. Review the Usage and limits documentation for details on limit notification settings.
How to structure workloads to stay under Claude throughput caps
To eliminate requests_per_minute and tokens_per_minute errors without relying solely on retry loops, manage your application's concurrency and token reservations:
- Choose the appropriate key type: A project key (
vz_ai_...) defaults to 120 requests and 200,000 tokens a minute with 25 concurrent requests in flight. It can be raised up to 600 requests, 2,000,000 tokens a minute, and 50 concurrent requests. Account keys (vz_sk_...) provide 300 requests and 400,000 tokens a minute with 50 concurrent requests. - Explicitly pass
max_tokens: When omitted,max_tokensdefaults to 1,024 (up to 16,384 per call). Before dispatching each request, the gateway reserves balance based on its worst-case calculation. Specifying a realisticmax_tokensensures you do not trigger pre-execution balance failures or tie up available concurrency. - Account for long prompts: Claude Haiku 5.5 (
vibiz/smart-mini) applies higher tier pricing above 100,000 input tokens. When submitting large context windows, batch prompts sequentially rather than issuing multiple parallel requests against the token-per-minute cap. - Throttle client-side queues: Restrict your application worker pools to stay below your key's concurrency limit (25 for standard project keys, 50 for account keys; a call over it waits up to 60 s for a slot before a 429).
To monitor live token volumes across tools, models, and end users, run:
vibiz usage --period 24h --by modelTo configure key quotas, raise spending thresholds, or set up automated credit replenishment, review our guides on AI and Billing and credits.
Frequently asked questions
What is the difference between a project key and an account key rate limit?
An account key allows 300 requests and 400,000 tokens per minute with 50 in-flight calls. A project key defaults to 120 requests and 200,000 tokens per minute with 25 in-flight calls, but an owner can raise its limits up to 600 requests, 2,000,000 tokens, and 50 in-flight calls.
Why do I get a 429 rate limit error when my request volume is low?
An HTTP 429 status code is returned for both throughput limits and spending caps. Inspect `error.details.reason`: values such as `daily_spend` (account ceiling) or `project_budget` (project ceiling) indicate a financial limit has been reached rather than a per-minute request spike.
At what time do daily spend limits reset?
All daily spend ceilings and project AI budgets reset at midnight UTC (00:00:00Z). The reset timestamp is also provided in usage payloads under `limits.resets_at`.
Can Claude model calls switch to another model automatically during rate limits?
No. thelastapi never substitutes another model when your requested model encounters rate limits or availability issues. The gateway returns an explicit error code so your application maintains predictable output and cost.