# JSON mode and structured output with any model

> Use JSON mode and structured output across OpenAI, Claude, Gemini, and DeepSeek via thelastapi's unified, OpenAI-compatible AI gateway.

**You can get clean, structured JSON from any major language model by calling the OpenAI-compatible gateway at `/v1/ai` with standard JSON parameters.** Because thelastapi accepts the OpenAI chat completions shape and routes calls across hundreds of models, you keep your existing code, use the standard OpenAI client or plain HTTP, and request structured JSON from GPT, Claude, Gemini, or DeepSeek without rewriting your parser.

When your application depends on reliable fields—like parsing an invoice, extracting CRM leads, or returning strict status codes—freeform text answers break your downstream code. Here is how to configure JSON mode and structured output, route calls across different models, and handle limits cleanly.

## How JSON mode works across labs

Every lab handles structured output slightly differently under the hood, but an OpenAI-compatible gateway removes that difference for your code. You pass `response_format` directly to `POST /v1/ai/chat/completions`:

- For raw JSON mode, set `response_format: { "type": "json_object" }` and tell the model in your system or user message to output JSON.
- For strict structured outputs, define your expected JSON schema inside `response_format`.

You can send this request to a unified thelastapi model tier—such as `vibiz/fast` (running GPT-6 Luna) or `vibiz/smart` (running Claude Sonnet 5.5)—or directly to any lab model by its own identifier (like `anthropic/claude-opus-5.5` or `moonshotai/kimi-k3.1-pro`). 

As explained in [how a unified AI API works](https://thelastapi.dev/blog/one-api-key-gpt-claude-gemini), thelastapi never silently swaps one model for another. If you ask for a model, exactly that model runs. If a provider is down, the request fails with `503 model_unavailable` or `503 provider_unavailable` rather than silently degrading output quality.

## Code: JSON mode using the OpenAI SDK

When you set up an app with the CLI or configure your keys, your environment receives `VIBIZ_AI_BASE_URL` and `VIBIZ_AI_API_KEY`, along with aliases `OPENAI_BASE_URL` and `OPENAI_API_KEY`. The official OpenAI SDK connects directly:

```ts
import OpenAI from 'openai';

const openai = new OpenAI({
  baseURL: process.env.VIBIZ_AI_BASE_URL,
  apiKey: process.env.VIBIZ_AI_API_KEY,
});

interface LeadExtraction {
  company: string;
  contact_name: string;
  role: string;
  budget_cents: number;
}

async function extractLead(rawEmail: string): Promise<LeadExtraction> {
  const response = await openai.chat.completions.create({
    model: 'vibiz/fast',
    response_format: { type: 'json_object' },
    messages: [
      {
        role: 'system',
        content: 'You extract sales lead data. Output valid JSON with keys: company, contact_name, role, budget_cents.',
      },
      {
        role: 'user',
        content: rawEmail,
      },
    ],
  });

  const content = response.choices[0]?.message.content ?? '{}';
  return JSON.parse(content) as LeadExtraction;
}
```

If you are using `@vibiz/app` inside your project, you can pass credentials straight to the OpenAI client without manually referencing individual environment variable names:

```ts
import OpenAI from 'openai';
import { vibiz } from '@vibiz/app';

const openai = new OpenAI(vibiz.ai);
```

For more architectural patterns involving gateways and frameworks, see [using LangChain with a multi-model gateway](https://thelastapi.dev/blog/langchain-multi-model-gateway).

## Making HTTP calls directly with curl

You can make structured output calls with standard HTTP clients from any language. Here is a `curl` example requesting JSON output from DeepSeek V4.1 Flash via `vibiz/lite`:

```bash
curl https://thelastapi.dev/v1/ai/chat/completions \
  -H "Authorization: Bearer $VIBIZ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "vibiz/lite",
    "response_format": { "type": "json_object" },
    "messages": [
      {
        "role": "system",
        "content": "Extract items into a JSON array of objects with keys: name, count."
      },
      {
        "role": "user",
        "content": "We bought two monitors and five keyboards."
      }
    ]
  }'
```

Every returned JSON response includes the standard choices array alongside metadata:
- `served_model`: Confirms the exact lab model that ran the call.
- `cost_micro_usd`: States the exact cost of that specific call in micro-USD.

## Comparing models for structured JSON tasks

Structured outputs are typically token-dense because schemas, keys, and values repeat across every response. Choosing the right tier balances schema precision against cost:

| Tier | Underlying lab model | Best suited for | Price per million tokens (in / out) |
| --- | --- | --- | --- |
| `vibiz/fast` | GPT-6 Luna (OpenAI) | Chat, tool calls, JSON extraction | $0.10 / $0.50 |
| `vibiz/lite` | DeepSeek V4.1 Flash (DeepSeek) | High-volume tasks, 1M token context | $0.14 / $0.42 |
| `vibiz/smart-mini` | Claude Haiku 5.5 (Anthropic) | Fast, structured data extraction | $0.10 / $0.50 |
| `vibiz/balanced` | Gemini 3.8 Flash (Google) | Balanced quality, 1M token context | $0.825 / $4.125 |
| `vibiz/smart` | Claude Sonnet 5.5 (Anthropic) | Complex nested schemas, reasoning | $2 / $10 |
| `vibiz/pro` | Claude Opus 5.5 (Anthropic) | Long agent executions, edge cases | $4 / $20 |

Beyond these tiers, you can call any model in the catalog using its lab-assigned ID (e.g., `moonshotai/kimi-k3.1-pro` or `anthropic/claude-opus-5.5`). You can inspect pricing and find models via the CLI:

```bash
vibiz ai models kimi
```

Review our guide on [AI usage dashboards per model](https://thelastapi.dev/blog/ai-usage-dashboard-per-model) to learn how to track token usage and costs per model tier.

## Handling budgets, rate limits, and errors

When running structured extractions over large datasets, requests run against concurrency, token, and daily budget limits:

- **Token caps**: `max_tokens` is 1,024 when omitted, and can be set up to 16,384 depending on the model.
- **Account rate limits**: Account keys (`vz_sk_...`) allow up to 300 requests and 400,000 tokens per minute (50 in-flight). Project keys (`vz_ai_...`) allow 120 requests and 200,000 tokens per minute (25 in-flight).
- **Daily spend limits**: Default account spend is capped at $5 per UTC day. Project budgets default to $1 per UTC day.

If your code exceeds these budgets, the API returns standard OpenAI-shaped errors with HTTP `429`:

```json
{
  "error": {
    "code": "rate_limited",
    "message": "Project budget exceeded.",
    "type": "invalid_request_error",
    "param": null,
    "details": {
      "reason": "project_budget"
    }
  }
}
```

When an account exhausts prepaid credits, requests fail with HTTP `402`:

```json
{
  "error": {
    "code": "payment_required",
    "message": "This account is out of credits. Add credits to keep using paid calls.",
    "details": {
      "reason": "insufficient_balance"
    }
  }
}
```

A project owner can adjust daily project budgets via the CLI:

```bash
vibiz ai budget 15 --project prj_123
```

You can view live consumption by tool or model directly from the command line:

```bash
vibiz usage --period 24h --by model
```

To learn more about spending policies, check the [AI documentation](https://thelastapi.dev/docs/ai), the [Usage reference](https://thelastapi.dev/docs/usage), and the [Billing guide](https://thelastapi.dev/docs/billing).

## Next steps

To start generating structured output with any model, initialize a project with AI capabilities using `vibiz init "Acme Parser" --resources ai`, run `vibiz dev` to pull your development credentials, and point your OpenAI SDK client to the gateway.

## Frequently asked questions

### How do I force JSON output if a model outputs markdown backticks?

Pass `response_format: { type: "json_object" }` on your request and state explicitly in your system message that the model must return raw JSON.

### Can I use your gateway with my own Google Gemini API key for structured output?

Yes, you can set your own key via `vibiz ai keys set google` or `PUT /v1/ai-keys/google`, which routes all Google models to your personal account at no platform markup.

### What happens if I send an invalid model name in my JSON request?

The gateway rejects unknown models immediately with an HTTP 400 `invalid_request` error naming the `model` parameter, preventing accidental fallback calls.

### How can I track JSON extraction costs per individual end user?

Pass your internal identifier in the `user` field or the `X-Vibiz-User` header; the usage dashboard will break down token spend and costs per user.

---

Published 2026-11-15 · https://thelastapi.dev/blog/json-mode-structured-output-any-model
