Streaming chat completions with server-sent events through one API
Stream chat completions from GPT, Claude, Gemini, and DeepSeek via standard Server-Sent Events through one unified OpenAI-compatible endpoint.
On this page 7 sections
You can stream chat completions from GPT, Claude, Gemini, DeepSeek, and over 400 other models through a single Server-Sent Events (SSE) endpoint. thelastapi provides an OpenAI-compatible gateway at /v1/ai/chat/completions where passing "stream": true yields standardized chunk deltas across every supported lab. Instead of writing custom streaming handlers for each provider's protocol, you keep a single client and switch models by identifier.
This guide walks through configuring SSE streams across multiple model families, handling stream chunks, reading mid-stream errors, and managing stream concurrency limits.
How unified streaming completions work
Every model lab has traditionally designed its own streaming format or wire event protocol. Anthropic uses event-based message deltas, Google uses its own streaming representation, and other labs follow variations of HTTP chunking. Writing frontend or backend parsers for each provider wastes engineering time.
thelastapi normalizes these streams at /v1/ai/chat/completions. When you make an HTTP request with "stream": true, the gateway connects to the requested lab, translates the response chunks in real time, and emits standard SSE messages formatted as:
data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"Hello"},"finish_reason":null}]}
data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":" world"},"finish_reason":null}]}
data: [DONE]You can target a general task tier (vibiz/fast, vibiz/smart, vibiz/balanced) or call any lab model directly by id (such as anthropic/claude-opus-5.5 or moonshotai/kimi-k3.1-pro). In every case, the gateway runs the exact model specified; it never substitutes a different model behind your back. To learn more about unifying multiple providers under one key, read about how a unified AI API works.
Streaming with the OpenAI SDK in TypeScript
Because the gateway adheres to the OpenAI specification, you do not need third-party wrappers or specialized streaming SDKs. The standard openai library consumes streams natively across all labs.
First, set up your project environment. If you run the CLI, you can initialize resources directly:
vibiz init "Streaming App" --resources ai
vibiz devThis exports VIBIZ_AI_BASE_URL and VIBIZ_AI_API_KEY (along with their OPENAI_* aliases). In your application code, instantiate the client and stream responses from Claude, Gemini, and GPT without altering the consumption loop:
import OpenAI from 'openai';
const openai = new OpenAI({
baseURL: process.env.VIBIZ_AI_BASE_URL,
apiKey: process.env.VIBIZ_AI_API_KEY,
});
async function streamResponse(model: string, prompt: string) {
const stream = await openai.chat.completions.create({
model,
messages: [{ role: 'user', content: prompt }],
stream: true,
});
for await (const chunk of stream) {
const text = chunk.choices[0]?.delta?.content || '';
process.stdout.write(text);
}
process.stdout.write('\n');
}
// Stream Anthropic Claude Sonnet 5.5
await streamResponse('vibiz/smart', 'Explain TCP in two sentences.');
// Stream Google Gemini 3.8 Flash
await streamResponse('vibiz/balanced', 'Explain UDP in two sentences.');
// Stream DeepSeek V4.1 Flash
await streamResponse('vibiz/lite', 'Explain DNS in two sentences.');If your application uses the @vibiz/app package, you can pass vibiz.ai directly to new OpenAI(vibiz.ai) without managing individual environment variables. Review the @vibiz/app documentation for runtime details.
Streaming via raw HTTP and curl
You can consume SSE streams from any HTTP client or terminal environment using curl. For more command-line patterns across models, see our curl chat completion examples.
To receive streamed tokens from Anthropic Claude Sonnet 5.5 (vibiz/smart), execute:
curl -N https://thelastapi.dev/v1/ai/chat/completions \
-H "Authorization: Bearer $VIBIZ_AI_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "vibiz/smart",
"messages": [{"role": "user", "content": "List three primary colors."}],
"stream": true
}'The -N flag disables output buffering in curl, printing each SSE data: chunk as soon as the gateway forwards it. The stream closes after the data: [DONE] signal.
From a command prompt, you can also stream responses without writing code using the vibiz ai CLI command:
vibiz ai "Write a haiku about deployment" --model vibiz/fastHow error handling works mid-stream
When an error occurs before output generation starts—such as an invalid authentication key (401) or a malformed JSON payload (400)—the gateway returns a standard HTTP error status code immediately.
When a failure happens mid-stream (for example, if an upstream lab disconnects, context limits are exceeded, or you exhaust account limits mid-generation), HTTP status headers have already been sent as 200 OK. The gateway handles this by streaming a structured error object as the final SSE event instead of the usual [DONE] signal:
data: {"error":{"code":"model_unavailable","message":"Upstream lab disconnected.","type":"provider_error"}}When building custom SSE consumers, always inspect the incoming JSON payload. If the error property is present on a received chunk, terminate the stream and surface the error to your application layer.
Common error codes to prepare for:
model_unavailable(503): The specific model is down. thelastapi never quietly routes your request to a different model tier. Checkdetails.retry_after_secondsor prompt the user to retry.rate_limited(429): You exceeded rate limits or budgets. Thedetails.reasonfield specifies whether the trigger wasdaily_spend,project_budget,requests_per_minute,tokens_per_minute, orconcurrency.content_filtered(400): Upstream safety filters refused the generation.context_length_exceeded(400): The prompt plus generated output exceeded the window limit.
Model tiers and streaming pricing
Because each thelastapi model points to a stable lab model, you get predictable performance and pricing while maintaining standard streaming interfaces.
| Model Tier | Upstream Lab Engine | Best Suited For | Input / Output Price (Per 1M tokens) |
|---|---|---|---|
vibiz/lite | DeepSeek V4.1 Flash | Fast lightweight chat, 1M context | $0.14 / $0.42 |
vibiz/fast | GPT-6 Luna (OpenAI) | Default chat, JSON output, tool calling | $0.10 / $0.50 |
vibiz/smart-mini | Claude Haiku 5.5 (Anthropic) | Fast coding and concise writing | $0.10 / $0.50 |
vibiz/balanced | Gemini 3.8 Flash (Google) | High context tasks, balanced quality | $0.825 / $4.125 |
vibiz/reasoning | GPT-6.1 Terra (OpenAI) | Logic, planning, step-by-step thinking | $1.00 / $5.00 |
vibiz/smart | Claude Sonnet 5.5 (Anthropic) | Difficult programming, complex agents | $2.00 / $10.00 |
vibiz/pro | Claude Opus 5.5 (Anthropic) | Maximum capability tier | $4.00 / $20.00 |
For broader catalog access, review our tutorial on how to access multiple AI models with a single API.
Managing streaming concurrency and limits
Streaming connections hold HTTP sockets open until token generation finishes. Understanding concurrency caps prevents unexpected connection throttling under heavy traffic.
- In-Flight Concurrency: An account key (
vz_sk_...) allows up to 50 concurrent in-flight requests. A project key (vz_ai_...) defaults to 25 in-flight requests (scalable up to 50). - Throughput Limits: An account key allows 300 requests and 400,000 tokens per minute. A project key allows 120 requests and 200,000 tokens per minute (configurable up to 600 requests and 2,000,000 tokens per minute).
- Spend Safeguards: Account spend is capped by default at $5 per UTC day, and individual projects have a $1 default daily budget. Each streaming request's worst-case token ceiling (
max_tokens, which defaults to 1,024 and caps at 16,384) is checked against your balance before the call starts.
If a project exceeds its daily allowance during streaming workloads, requests return 429 with details.reason: "project_budget". The account owner can adjust this limit in the console or CLI:
vibiz ai budget 15 --project prj_123You can track your streaming usage by model across any time window using the CLI:
vibiz usage --period 24h --by modelTo dive deeper into spending rules, parameters, and token ceilings, read the complete AI documentation.
Frequently asked questions
How does thelastapi handle SSE token streams across different model providers?
The gateway translates each upstream provider's native streaming output into standard OpenAI-compatible Server-Sent Events chunks (`data: {"choices":[{"delta":{"content":"..."}}]}`). You use the exact same client parsing logic regardless of whether you call OpenAI, Anthropic, Google, or DeepSeek.
What happens if an error occurs while a model is streaming tokens?
Because HTTP headers have already been transmitted, mid-stream errors arrive as a final SSE event containing `data: {"error": {...}}` instead of the standard `data: [DONE]` signal. Your stream consumer should verify that incoming chunks do not contain an error property before writing output.
Can I use custom end-user identifiers with streaming calls for budget tracking?
Yes. You can pass your application's user identifier in the `user` field of the request payload or as the `X-Vibiz-User` HTTP header. The gateway tracks token spend per user, enabling you to inspect per-user consumption with `vibiz usage --by end_user`.
What is the default maximum output length when streaming a response?
The `max_tokens` parameter defaults to 1,024 tokens if omitted from your request body, with a maximum allowable value of 16,384 tokens depending on the chosen model's architecture.