LangChain with a multi-model OpenAI-compatible gateway
Connect LangChain to multiple AI models using an OpenAI-compatible gateway with a single base URL, one API key, and zero model swapping.
On this page 8 sections
You can connect LangChain to multiple AI labs by pointing LangChain's ChatOpenAI class at an OpenAI-compatible gateway URL and passing your gateway API key. thelastapi provides an OpenAI-compatible endpoint at /v1/ai that lets you call models from OpenAI, Anthropic, Google, DeepSeek, Moonshot AI, and Meta without installing separate lab libraries. You keep your existing LangChain chains, prompts, and tool configurations while switching between models simply by updating the model name string.
This guide walks through configuring LangChain in Python and TypeScript, routing to different lab models through one interface, and monitoring costs and rate limits.
How a multi-model gateway works with LangChain
LangChain includes native wrappers for many individual model providers, but managing separate packages creates maintenance friction. Each provider requires its own client initialization, credential handling, and version management. If you want to benchmark an Anthropic model against a Google model, you usually need separate dependencies, separate account dashboards, and separate billing cards.
An OpenAI-compatible gateway changes this workflow. Because LangChain's ChatOpenAI integration expects the standard OpenAI Chat Completions REST format, you can route requests through any gateway that implements that contract.
When your LangChain code executes a prompt, the gateway receives the request at /v1/ai/chat/completions, runs the exact model you specified, and streams back the response in standard OpenAI JSON or SSE chunks. thelastapi never silently swaps models if one provider encounters downtime; if a model is unreachable, the API returns a 503 with a model_unavailable or provider_unavailable error code. To understand the underlying transport architecture, see our guide on how a unified AI API works.
Setting up your credentials and environment
To use the gateway with LangChain, you need a project AI key or account key. You can initialize a project using the CLI:
vibiz init "LangChain Assistant" --resources ai
vibiz devThe vibiz dev command provisions your local development environment and populates your local environment file with the necessary connection details. It provides VIBIZ_AI_BASE_URL and VIBIZ_AI_API_KEY, alongside the standard OPENAI_BASE_URL and OPENAI_API_KEY aliases.
The standard endpoint for AI calls is:
- Base URL:
https://thelastapi.dev/v1/ai - Key Format:
vz_ai_...(for a project AI key) orvz_sk_...(for an account key)
If you are using @vibiz/app within an existing application, the SDK exposes your credentials directly via vibiz.ai. You can read more about managing access tokens and security boundaries in the Keys and environments documentation.
Configuring LangChain to call multiple models
Because the gateway supports standard OpenAI request structures, you instantiate LangChain's ChatOpenAI client directly. You do not need to install provider-specific integrations like @langchain/anthropic or langchain-google-genai.
Python setup
Install the standard LangChain OpenAI package:
pip install langchain-openaiPoint the client to the gateway endpoint:
import os
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage
# Initialize the client pointing to the gateway
llm = ChatOpenAI(
base_url=os.environ.get("VIBIZ_AI_BASE_URL", "https://thelastapi.dev/v1/ai"),
api_key=os.environ.get("VIBIZ_AI_API_KEY"),
model="vibiz/smart"
)
response = llm.invoke([
HumanMessage(content="Explain distributed transactions in three sentences.")
])
print(response.content)TypeScript setup
Install the JavaScript package:
npm install @langchain/openaiInstantiate ChatOpenAI using the configuration parameter:
import { ChatOpenAI } from '@langchain/openai';
import { HumanMessage } from '@langchain/core/messages';
const model = new ChatOpenAI({
modelName: 'vibiz/balanced',
configuration: {
baseURL: process.env.VIBIZ_AI_BASE_URL || 'https://thelastapi.dev/v1/ai',
apiKey: process.env.VIBIZ_AI_API_KEY,
},
});
const res = await model.invoke([
new HumanMessage('Generate three distinct naming ideas for an uptime monitoring product.'),
]);
console.log(res.content);For teams considering whether to connect directly to each provider or standardize on an adapter layer, read our breakdown on using OpenAI directly vs a model gateway.
Choosing between stable tiers and direct model IDs
thelastapi provides two ways to address models in your LangChain components: functional model tiers and explicit lab identifiers.
1. Functional tiers
Functional tiers are stable model names mapped to specific lab versions. When a lab releases an update, the underlying model is upgraded, but your LangChain configuration string remains the same. Every API response includes a served_model field so you always know which exact checkpoint handled the generation.
| Model Tier | Underlying Model | Primary Use Case | Price per 1M tokens (In / Out) |
|---|---|---|---|
vibiz/fast | GPT-6 Luna (OpenAI) | Chat, tool calling, JSON parsing | $0.10 / $0.50 |
vibiz/lite | DeepSeek V4.1 Flash | Cost-efficient generation, 1M context | $0.14 / $0.42 |
vibiz/smart-mini | Claude Haiku 5.5 (Anthropic) | Fast coding and technical writing | $0.10 / $0.50 |
vibiz/balanced | Gemini 3.8 Flash (Google) | Balanced quality, 1M context | $0.825 / $4.125 |
vibiz/reasoning | GPT-6.1 Terra (OpenAI) | Multi-step logic, math, planning | $1.00 / $5.00 |
vibiz/smart | Claude Sonnet 5.5 (Anthropic) | Complex coding, long chains | $2.00 / $10.00 |
vibiz/pro | Claude Opus 5.5 (Anthropic) | Deep reasoning and evaluation | $4.00 / $20.00 |
2. Direct model IDs
You can also pass exact provider IDs directly to the model parameter. Over 400 models from OpenAI, Anthropic, Google, DeepSeek, Moonshot AI, Mistral, Qwen, and Meta are available. For example:
# Call Claude Opus directly
opus = ChatOpenAI(
base_url="https://thelastapi.dev/v1/ai",
api_key=os.environ["VIBIZ_AI_API_KEY"],
model="anthropic/claude-opus-5.5"
)
# Call Moonshot Kimi directly
kimi = ChatOpenAI(
base_url="https://thelastapi.dev/v1/ai",
api_key=os.environ["VIBIZ_AI_API_KEY"],
model="moonshotai/kimi-k3.1-pro"
)If you specify an invalid or mistyped model string, the gateway returns a 400 invalid_request with param: "model". It will never default to a fallback model without your consent.
Managing vector embeddings and tool calling
LangChain workflows often require vector embeddings and function execution alongside text generation.
Vector embeddings
You can use standard LangChain embedding classes pointing to /v1/ai/embeddings:
from langchain_openai import OpenAIEmbeddings
embeddings = OpenAIEmbeddings(
base_url="https://thelastapi.dev/v1/ai",
api_key=os.environ["VIBIZ_AI_API_KEY"],
model="vibiz/embed" # text-embedding-3-small, 1536 dims, $0.02 / 1M tokens
)
vector = embeddings.embed_query("Query string to vectorize")For higher dimensionality tasks, vibiz/embed-large maps to text-embedding-3-large (3,072 dimensions) at $0.13 per million tokens. The embedding batch limit accepts up to 2,048 inputs per request.
Tool calling
The gateway supports OpenAI-formatted function calling across compatible models, including vibiz/fast and vibiz/smart:
from langchain_core.tools import tool
@tool
def lookup_customer(customer_id: str) -> str:
"""Retrieves account details for a customer."""
return f"Active customer record for {customer_id}"
model_with_tools = llm.bind_tools([lookup_customer])
result = model_with_tools.invoke("Look up customer u_123")
print(result.tool_calls)For frontend and full-stack projects using @ai-sdk/openai instead of LangChain, read how to use @ai-sdk/openai with Claude, Gemini and more.
Setting spend limits and tracking usage
Unbounded LangChain agent loops can quickly exhaust API balances. thelastapi includes safety controls at the platform layer to prevent unexpected overages:
- Daily safety cap: Accounts default to a $5 daily spending ceiling across AI and background actions. Account owners can raise this up to $1,000 via Billing or using the CLI command
vibiz billing limit <dollars>. - Project AI budgets: Each project defaults to a $1 daily AI limit. When reached, requests receive a
429 rate_limitedresponse withdetails.reason: "project_budget". You can modify this with:Terminalvibiz ai budget 10 --project prj_123 - End-user attribution: You can pass an end-user identifier in your LangChain calls by appending the
userfield or theX-Vibiz-UserHTTP header. This enablesend_user_daily_budget_usdprotections and segments spend logs.
Every JSON completion response returns cost_micro_usd, which represents the exact micro-dollar cost of the request. You can check current spend figures and token counts directly from your shell:
# View usage by model over the last 7 days
vibiz usage --period 7d --by model
# View usage segmented by end user
vibiz usage --by end_user --project prj_123Detailed pricing mechanics, balance top-ups, and auto-recharge settings are documented in Billing and credits and the AI overview.
Next steps
To integrate your LangChain pipeline with the gateway, generate a project key using vibiz add ai or configure your existing development credentials. You can inspect all currently available models and real-time token pricing at any time using vibiz ai models --all.
Frequently asked questions
Can I use LangChain memory components like ConversationBufferMemory through the gateway?
Yes. Gateway calls use the standard OpenAI chat completions endpoint, so all standard LangChain message abstractions and memory classes pass their message arrays as usual.
What happens if a specific AI model is down while my LangChain pipeline runs?
The gateway returns a 503 error status with `model_unavailable` or `provider_unavailable` instead of silently routing your prompt to a different model. Your LangChain error handling catches this standard API error so your application can retry or notify the user.
Can I route Google Gemini requests through my own Google key?
Yes. You can register your Google API key using `GEMINI_API_KEY=... vibiz ai keys set google`, which causes Gemini models (including `vibiz/balanced`) to execute against your own Google account at zero billing charge on the gateway.
How do rate limits work when calling multiple models through LangChain?
Project keys allow 120 requests and 200,000 tokens per minute by default, while account keys allow 300 requests and 400,000 tokens per minute. If you exceed limits or hit your daily budget ceiling, the gateway returns a 429 status code with a descriptive `details.reason` field.