# Streaming chat completions with server-sent events through one API

> Stream chat completions from GPT, Claude, Gemini, and DeepSeek via standard Server-Sent Events through one unified OpenAI-compatible endpoint.

**You can stream chat completions from GPT, Claude, Gemini, DeepSeek, and over 400 other models through a single Server-Sent Events (SSE) endpoint.** thelastapi provides an OpenAI-compatible gateway at `/v1/ai/chat/completions` where passing `"stream": true` yields standardized chunk deltas across every supported lab. Instead of writing custom streaming handlers for each provider's protocol, you keep a single client and switch models by identifier.

This guide walks through configuring SSE streams across multiple model families, handling stream chunks, reading mid-stream errors, and managing stream concurrency limits.

## How unified streaming completions work

Every model lab has traditionally designed its own streaming format or wire event protocol. Anthropic uses event-based message deltas, Google uses its own streaming representation, and other labs follow variations of HTTP chunking. Writing frontend or backend parsers for each provider wastes engineering time.

thelastapi normalizes these streams at `/v1/ai/chat/completions`. When you make an HTTP request with `"stream": true`, the gateway connects to the requested lab, translates the response chunks in real time, and emits standard SSE messages formatted as:

```text
data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"Hello"},"finish_reason":null}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":" world"},"finish_reason":null}]}

data: [DONE]
```

You can target a general task tier (`vibiz/fast`, `vibiz/smart`, `vibiz/balanced`) or call any lab model directly by id (such as `anthropic/claude-opus-5.5` or `moonshotai/kimi-k3.1-pro`). In every case, the gateway runs the exact model specified; it never substitutes a different model behind your back. To learn more about unifying multiple providers under one key, read about [how a unified AI API works](https://thelastapi.dev/blog/one-api-key-gpt-claude-gemini).

## Streaming with the OpenAI SDK in TypeScript

Because the gateway adheres to the OpenAI specification, you do not need third-party wrappers or specialized streaming SDKs. The standard `openai` library consumes streams natively across all labs.

First, set up your project environment. If you run the CLI, you can initialize resources directly:

```bash
vibiz init "Streaming App" --resources ai
vibiz dev
```

This exports `VIBIZ_AI_BASE_URL` and `VIBIZ_AI_API_KEY` (along with their `OPENAI_*` aliases). In your application code, instantiate the client and stream responses from Claude, Gemini, and GPT without altering the consumption loop:

```ts
import OpenAI from 'openai';

const openai = new OpenAI({
  baseURL: process.env.VIBIZ_AI_BASE_URL,
  apiKey: process.env.VIBIZ_AI_API_KEY,
});

async function streamResponse(model: string, prompt: string) {
  const stream = await openai.chat.completions.create({
    model,
    messages: [{ role: 'user', content: prompt }],
    stream: true,
  });

  for await (const chunk of stream) {
    const text = chunk.choices[0]?.delta?.content || '';
    process.stdout.write(text);
  }
  process.stdout.write('\n');
}

// Stream Anthropic Claude Sonnet 5.5
await streamResponse('vibiz/smart', 'Explain TCP in two sentences.');

// Stream Google Gemini 3.8 Flash
await streamResponse('vibiz/balanced', 'Explain UDP in two sentences.');

// Stream DeepSeek V4.1 Flash
await streamResponse('vibiz/lite', 'Explain DNS in two sentences.');
```

If your application uses the `@vibiz/app` package, you can pass `vibiz.ai` directly to `new OpenAI(vibiz.ai)` without managing individual environment variables. Review the [@vibiz/app documentation](https://thelastapi.dev/docs/app-sdk) for runtime details.

## Streaming via raw HTTP and curl

You can consume SSE streams from any HTTP client or terminal environment using `curl`. For more command-line patterns across models, see our [curl chat completion examples](https://thelastapi.dev/blog/curl-chat-completions-different-models).

To receive streamed tokens from Anthropic Claude Sonnet 5.5 (`vibiz/smart`), execute:

```bash
curl -N https://thelastapi.dev/v1/ai/chat/completions \
  -H "Authorization: Bearer $VIBIZ_AI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "vibiz/smart",
    "messages": [{"role": "user", "content": "List three primary colors."}],
    "stream": true
  }'
```

The `-N` flag disables output buffering in curl, printing each SSE `data:` chunk as soon as the gateway forwards it. The stream closes after the `data: [DONE]` signal.

From a command prompt, you can also stream responses without writing code using the `vibiz ai` CLI command:

```bash
vibiz ai "Write a haiku about deployment" --model vibiz/fast
```

## How error handling works mid-stream

When an error occurs before output generation starts—such as an invalid authentication key (`401`) or a malformed JSON payload (`400`)—the gateway returns a standard HTTP error status code immediately.

When a failure happens *mid-stream* (for example, if an upstream lab disconnects, context limits are exceeded, or you exhaust account limits mid-generation), HTTP status headers have already been sent as `200 OK`. The gateway handles this by streaming a structured error object as the final SSE event instead of the usual `[DONE]` signal:

```text
data: {"error":{"code":"model_unavailable","message":"Upstream lab disconnected.","type":"provider_error"}}
```

When building custom SSE consumers, always inspect the incoming JSON payload. If the `error` property is present on a received chunk, terminate the stream and surface the error to your application layer.

Common error codes to prepare for:

- `model_unavailable` (503): The specific model is down. thelastapi never quietly routes your request to a different model tier. Check `details.retry_after_seconds` or prompt the user to retry.
- `rate_limited` (429): You exceeded rate limits or budgets. The `details.reason` field specifies whether the trigger was `daily_spend`, `project_budget`, `requests_per_minute`, `tokens_per_minute`, or `concurrency`.
- `content_filtered` (400): Upstream safety filters refused the generation.
- `context_length_exceeded` (400): The prompt plus generated output exceeded the window limit.

## Model tiers and streaming pricing

Because each thelastapi model points to a stable lab model, you get predictable performance and pricing while maintaining standard streaming interfaces.

| Model Tier | Upstream Lab Engine | Best Suited For | Input / Output Price (Per 1M tokens) |
| --- | --- | --- | --- |
| `vibiz/lite` | DeepSeek V4.1 Flash | Fast lightweight chat, 1M context | $0.14 / $0.42 |
| `vibiz/fast` | GPT-6 Luna (OpenAI) | Default chat, JSON output, tool calling | $0.10 / $0.50 |
| `vibiz/smart-mini` | Claude Haiku 5.5 (Anthropic) | Fast coding and concise writing | $0.10 / $0.50 |
| `vibiz/balanced` | Gemini 3.8 Flash (Google) | High context tasks, balanced quality | $0.825 / $4.125 |
| `vibiz/reasoning` | GPT-6.1 Terra (OpenAI) | Logic, planning, step-by-step thinking | $1.00 / $5.00 |
| `vibiz/smart` | Claude Sonnet 5.5 (Anthropic) | Difficult programming, complex agents | $2.00 / $10.00 |
| `vibiz/pro` | Claude Opus 5.5 (Anthropic) | Maximum capability tier | $4.00 / $20.00 |

For broader catalog access, review our tutorial on [how to access multiple AI models with a single API](https://thelastapi.dev/blog/access-multiple-ai-models-single-api).

## Managing streaming concurrency and limits

Streaming connections hold HTTP sockets open until token generation finishes. Understanding concurrency caps prevents unexpected connection throttling under heavy traffic.

1. **In-Flight Concurrency**: An account key (`vz_sk_...`) allows up to 50 concurrent in-flight requests. A project key (`vz_ai_...`) defaults to 25 in-flight requests (scalable up to 50).
2. **Throughput Limits**: An account key allows 300 requests and 400,000 tokens per minute. A project key allows 120 requests and 200,000 tokens per minute (configurable up to 600 requests and 2,000,000 tokens per minute).
3. **Spend Safeguards**: Account spend is capped by default at $5 per UTC day, and individual projects have a $1 default daily budget. Each streaming request's worst-case token ceiling (`max_tokens`, which defaults to 1,024 and caps at 16,384) is checked against your balance before the call starts.

If a project exceeds its daily allowance during streaming workloads, requests return `429` with `details.reason: "project_budget"`. The account owner can adjust this limit in the console or CLI:

```bash
vibiz ai budget 15 --project prj_123
```

You can track your streaming usage by model across any time window using the CLI:

```bash
vibiz usage --period 24h --by model
```

To dive deeper into spending rules, parameters, and token ceilings, read the complete [AI documentation](https://thelastapi.dev/docs/ai).

## Frequently asked questions

### How does thelastapi handle SSE token streams across different model providers?

The gateway translates each upstream provider's native streaming output into standard OpenAI-compatible Server-Sent Events chunks (`data: {"choices":[{"delta":{"content":"..."}}]}`). You use the exact same client parsing logic regardless of whether you call OpenAI, Anthropic, Google, or DeepSeek.

### What happens if an error occurs while a model is streaming tokens?

Because HTTP headers have already been transmitted, mid-stream errors arrive as a final SSE event containing `data: {"error": {...}}` instead of the standard `data: [DONE]` signal. Your stream consumer should verify that incoming chunks do not contain an error property before writing output.

### Can I use custom end-user identifiers with streaming calls for budget tracking?

Yes. You can pass your application's user identifier in the `user` field of the request payload or as the `X-Vibiz-User` HTTP header. The gateway tracks token spend per user, enabling you to inspect per-user consumption with `vibiz usage --by end_user`.

### What is the default maximum output length when streaming a response?

The `max_tokens` parameter defaults to 1,024 tokens if omitted from your request body, with a maximum allowable value of 16,384 tokens depending on the chosen model's architecture.

---

Published 2026-11-15 · https://thelastapi.dev/blog/streaming-chat-completions-sse
