JSON mode and structured output with any model
Use JSON mode and structured output across OpenAI, Claude, Gemini, and DeepSeek via thelastapi's unified, OpenAI-compatible AI gateway.
On this page 7 sections
You can get clean, structured JSON from any major language model by calling the OpenAI-compatible gateway at /v1/ai with standard JSON parameters. Because thelastapi accepts the OpenAI chat completions shape and routes calls across hundreds of models, you keep your existing code, use the standard OpenAI client or plain HTTP, and request structured JSON from GPT, Claude, Gemini, or DeepSeek without rewriting your parser.
When your application depends on reliable fields—like parsing an invoice, extracting CRM leads, or returning strict status codes—freeform text answers break your downstream code. Here is how to configure JSON mode and structured output, route calls across different models, and handle limits cleanly.
How JSON mode works across labs
Every lab handles structured output slightly differently under the hood, but an OpenAI-compatible gateway removes that difference for your code. You pass response_format directly to POST /v1/ai/chat/completions:
- For raw JSON mode, set
response_format: { "type": "json_object" }and tell the model in your system or user message to output JSON. - For strict structured outputs, define your expected JSON schema inside
response_format.
You can send this request to a unified thelastapi model tier—such as vibiz/fast (running GPT-6 Luna) or vibiz/smart (running Claude Sonnet 5.5)—or directly to any lab model by its own identifier (like anthropic/claude-opus-5.5 or moonshotai/kimi-k3.1-pro).
As explained in how a unified AI API works, thelastapi never silently swaps one model for another. If you ask for a model, exactly that model runs. If a provider is down, the request fails with 503 model_unavailable or 503 provider_unavailable rather than silently degrading output quality.
Code: JSON mode using the OpenAI SDK
When you set up an app with the CLI or configure your keys, your environment receives VIBIZ_AI_BASE_URL and VIBIZ_AI_API_KEY, along with aliases OPENAI_BASE_URL and OPENAI_API_KEY. The official OpenAI SDK connects directly:
import OpenAI from 'openai';
const openai = new OpenAI({
baseURL: process.env.VIBIZ_AI_BASE_URL,
apiKey: process.env.VIBIZ_AI_API_KEY,
});
interface LeadExtraction {
company: string;
contact_name: string;
role: string;
budget_cents: number;
}
async function extractLead(rawEmail: string): Promise<LeadExtraction> {
const response = await openai.chat.completions.create({
model: 'vibiz/fast',
response_format: { type: 'json_object' },
messages: [
{
role: 'system',
content: 'You extract sales lead data. Output valid JSON with keys: company, contact_name, role, budget_cents.',
},
{
role: 'user',
content: rawEmail,
},
],
});
const content = response.choices[0]?.message.content ?? '{}';
return JSON.parse(content) as LeadExtraction;
}If you are using @vibiz/app inside your project, you can pass credentials straight to the OpenAI client without manually referencing individual environment variable names:
import OpenAI from 'openai';
import { vibiz } from '@vibiz/app';
const openai = new OpenAI(vibiz.ai);For more architectural patterns involving gateways and frameworks, see using LangChain with a multi-model gateway.
Making HTTP calls directly with curl
You can make structured output calls with standard HTTP clients from any language. Here is a curl example requesting JSON output from DeepSeek V4.1 Flash via vibiz/lite:
curl https://thelastapi.dev/v1/ai/chat/completions \
-H "Authorization: Bearer $VIBIZ_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "vibiz/lite",
"response_format": { "type": "json_object" },
"messages": [
{
"role": "system",
"content": "Extract items into a JSON array of objects with keys: name, count."
},
{
"role": "user",
"content": "We bought two monitors and five keyboards."
}
]
}'Every returned JSON response includes the standard choices array alongside metadata:
served_model: Confirms the exact lab model that ran the call.cost_micro_usd: States the exact cost of that specific call in micro-USD.
Comparing models for structured JSON tasks
Structured outputs are typically token-dense because schemas, keys, and values repeat across every response. Choosing the right tier balances schema precision against cost:
| Tier | Underlying lab model | Best suited for | Price per million tokens (in / out) |
|---|---|---|---|
vibiz/fast | GPT-6 Luna (OpenAI) | Chat, tool calls, JSON extraction | $0.10 / $0.50 |
vibiz/lite | DeepSeek V4.1 Flash (DeepSeek) | High-volume tasks, 1M token context | $0.14 / $0.42 |
vibiz/smart-mini | Claude Haiku 5.5 (Anthropic) | Fast, structured data extraction | $0.10 / $0.50 |
vibiz/balanced | Gemini 3.8 Flash (Google) | Balanced quality, 1M token context | $0.825 / $4.125 |
vibiz/smart | Claude Sonnet 5.5 (Anthropic) | Complex nested schemas, reasoning | $2 / $10 |
vibiz/pro | Claude Opus 5.5 (Anthropic) | Long agent executions, edge cases | $4 / $20 |
Beyond these tiers, you can call any model in the catalog using its lab-assigned ID (e.g., moonshotai/kimi-k3.1-pro or anthropic/claude-opus-5.5). You can inspect pricing and find models via the CLI:
vibiz ai models kimiReview our guide on AI usage dashboards per model to learn how to track token usage and costs per model tier.
Handling budgets, rate limits, and errors
When running structured extractions over large datasets, requests run against concurrency, token, and daily budget limits:
- Token caps:
max_tokensis 1,024 when omitted, and can be set up to 16,384 depending on the model. - Account rate limits: Account keys (
vz_sk_...) allow up to 300 requests and 400,000 tokens per minute (50 in-flight). Project keys (vz_ai_...) allow 120 requests and 200,000 tokens per minute (25 in-flight). - Daily spend limits: Default account spend is capped at $5 per UTC day. Project budgets default to $1 per UTC day.
If your code exceeds these budgets, the API returns standard OpenAI-shaped errors with HTTP 429:
{
"error": {
"code": "rate_limited",
"message": "Project budget exceeded.",
"type": "invalid_request_error",
"param": null,
"details": {
"reason": "project_budget"
}
}
}When an account exhausts prepaid credits, requests fail with HTTP 402:
{
"error": {
"code": "payment_required",
"message": "This account is out of credits. Add credits to keep using paid calls.",
"details": {
"reason": "insufficient_balance"
}
}
}A project owner can adjust daily project budgets via the CLI:
vibiz ai budget 15 --project prj_123You can view live consumption by tool or model directly from the command line:
vibiz usage --period 24h --by modelTo learn more about spending policies, check the AI documentation, the Usage reference, and the Billing guide.
Next steps
To start generating structured output with any model, initialize a project with AI capabilities using vibiz init "Acme Parser" --resources ai, run vibiz dev to pull your development credentials, and point your OpenAI SDK client to the gateway.
Frequently asked questions
How do I force JSON output if a model outputs markdown backticks?
Pass `response_format: { type: "json_object" }` on your request and state explicitly in your system message that the model must return raw JSON.
Can I use your gateway with my own Google Gemini API key for structured output?
Yes, you can set your own key via `vibiz ai keys set google` or `PUT /v1/ai-keys/google`, which routes all Google models to your personal account at no platform markup.
What happens if I send an invalid model name in my JSON request?
The gateway rejects unknown models immediately with an HTTP 400 `invalid_request` error naming the `model` parameter, preventing accidental fallback calls.
How can I track JSON extraction costs per individual end user?
Pass your internal identifier in the `user` field or the `X-Vibiz-User` header; the usage dashboard will break down token spend and costs per user.