thelastapi
Get started free

LangChain with a multi-model OpenAI-compatible gateway

Connect LangChain to multiple AI models using an OpenAI-compatible gateway with a single base URL, one API key, and zero model swapping.

On this page 8 sections

You can connect LangChain to multiple AI labs by pointing LangChain's ChatOpenAI class at an OpenAI-compatible gateway URL and passing your gateway API key. thelastapi provides an OpenAI-compatible endpoint at /v1/ai that lets you call models from OpenAI, Anthropic, Google, DeepSeek, Moonshot AI, and Meta without installing separate lab libraries. You keep your existing LangChain chains, prompts, and tool configurations while switching between models simply by updating the model name string.

This guide walks through configuring LangChain in Python and TypeScript, routing to different lab models through one interface, and monitoring costs and rate limits.

How a multi-model gateway works with LangChain

LangChain includes native wrappers for many individual model providers, but managing separate packages creates maintenance friction. Each provider requires its own client initialization, credential handling, and version management. If you want to benchmark an Anthropic model against a Google model, you usually need separate dependencies, separate account dashboards, and separate billing cards.

An OpenAI-compatible gateway changes this workflow. Because LangChain's ChatOpenAI integration expects the standard OpenAI Chat Completions REST format, you can route requests through any gateway that implements that contract.

When your LangChain code executes a prompt, the gateway receives the request at /v1/ai/chat/completions, runs the exact model you specified, and streams back the response in standard OpenAI JSON or SSE chunks. thelastapi never silently swaps models if one provider encounters downtime; if a model is unreachable, the API returns a 503 with a model_unavailable or provider_unavailable error code. To understand the underlying transport architecture, see our guide on how a unified AI API works.

Setting up your credentials and environment

To use the gateway with LangChain, you need a project AI key or account key. You can initialize a project using the CLI:

Terminal
vibiz init "LangChain Assistant" --resources ai
vibiz dev

The vibiz dev command provisions your local development environment and populates your local environment file with the necessary connection details. It provides VIBIZ_AI_BASE_URL and VIBIZ_AI_API_KEY, alongside the standard OPENAI_BASE_URL and OPENAI_API_KEY aliases.

The standard endpoint for AI calls is:

  • Base URL: https://thelastapi.dev/v1/ai
  • Key Format: vz_ai_... (for a project AI key) or vz_sk_... (for an account key)

If you are using @vibiz/app within an existing application, the SDK exposes your credentials directly via vibiz.ai. You can read more about managing access tokens and security boundaries in the Keys and environments documentation.

Configuring LangChain to call multiple models

Because the gateway supports standard OpenAI request structures, you instantiate LangChain's ChatOpenAI client directly. You do not need to install provider-specific integrations like @langchain/anthropic or langchain-google-genai.

Python setup

Install the standard LangChain OpenAI package:

Terminal
pip install langchain-openai

Point the client to the gateway endpoint:

python
import os
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage

# Initialize the client pointing to the gateway
llm = ChatOpenAI(
    base_url=os.environ.get("VIBIZ_AI_BASE_URL", "https://thelastapi.dev/v1/ai"),
    api_key=os.environ.get("VIBIZ_AI_API_KEY"),
    model="vibiz/smart"
)

response = llm.invoke([
    HumanMessage(content="Explain distributed transactions in three sentences.")
])
print(response.content)

TypeScript setup

Install the JavaScript package:

Terminal
npm install @langchain/openai

Instantiate ChatOpenAI using the configuration parameter:

TypeScript
import { ChatOpenAI } from '@langchain/openai';
import { HumanMessage } from '@langchain/core/messages';

const model = new ChatOpenAI({
  modelName: 'vibiz/balanced',
  configuration: {
    baseURL: process.env.VIBIZ_AI_BASE_URL || 'https://thelastapi.dev/v1/ai',
    apiKey: process.env.VIBIZ_AI_API_KEY,
  },
});

const res = await model.invoke([
  new HumanMessage('Generate three distinct naming ideas for an uptime monitoring product.'),
]);
console.log(res.content);

For teams considering whether to connect directly to each provider or standardize on an adapter layer, read our breakdown on using OpenAI directly vs a model gateway.

Choosing between stable tiers and direct model IDs

thelastapi provides two ways to address models in your LangChain components: functional model tiers and explicit lab identifiers.

1. Functional tiers

Functional tiers are stable model names mapped to specific lab versions. When a lab releases an update, the underlying model is upgraded, but your LangChain configuration string remains the same. Every API response includes a served_model field so you always know which exact checkpoint handled the generation.

Model TierUnderlying ModelPrimary Use CasePrice per 1M tokens (In / Out)
vibiz/fastGPT-6 Luna (OpenAI)Chat, tool calling, JSON parsing$0.10 / $0.50
vibiz/liteDeepSeek V4.1 FlashCost-efficient generation, 1M context$0.14 / $0.42
vibiz/smart-miniClaude Haiku 5.5 (Anthropic)Fast coding and technical writing$0.10 / $0.50
vibiz/balancedGemini 3.8 Flash (Google)Balanced quality, 1M context$0.825 / $4.125
vibiz/reasoningGPT-6.1 Terra (OpenAI)Multi-step logic, math, planning$1.00 / $5.00
vibiz/smartClaude Sonnet 5.5 (Anthropic)Complex coding, long chains$2.00 / $10.00
vibiz/proClaude Opus 5.5 (Anthropic)Deep reasoning and evaluation$4.00 / $20.00

2. Direct model IDs

You can also pass exact provider IDs directly to the model parameter. Over 400 models from OpenAI, Anthropic, Google, DeepSeek, Moonshot AI, Mistral, Qwen, and Meta are available. For example:

python
# Call Claude Opus directly
opus = ChatOpenAI(
    base_url="https://thelastapi.dev/v1/ai",
    api_key=os.environ["VIBIZ_AI_API_KEY"],
    model="anthropic/claude-opus-5.5"
)

# Call Moonshot Kimi directly
kimi = ChatOpenAI(
    base_url="https://thelastapi.dev/v1/ai",
    api_key=os.environ["VIBIZ_AI_API_KEY"],
    model="moonshotai/kimi-k3.1-pro"
)

If you specify an invalid or mistyped model string, the gateway returns a 400 invalid_request with param: "model". It will never default to a fallback model without your consent.

Managing vector embeddings and tool calling

LangChain workflows often require vector embeddings and function execution alongside text generation.

Vector embeddings

You can use standard LangChain embedding classes pointing to /v1/ai/embeddings:

python
from langchain_openai import OpenAIEmbeddings

embeddings = OpenAIEmbeddings(
    base_url="https://thelastapi.dev/v1/ai",
    api_key=os.environ["VIBIZ_AI_API_KEY"],
    model="vibiz/embed"  # text-embedding-3-small, 1536 dims, $0.02 / 1M tokens
)

vector = embeddings.embed_query("Query string to vectorize")

For higher dimensionality tasks, vibiz/embed-large maps to text-embedding-3-large (3,072 dimensions) at $0.13 per million tokens. The embedding batch limit accepts up to 2,048 inputs per request.

Tool calling

The gateway supports OpenAI-formatted function calling across compatible models, including vibiz/fast and vibiz/smart:

python
from langchain_core.tools import tool

@tool
def lookup_customer(customer_id: str) -> str:
    """Retrieves account details for a customer."""
    return f"Active customer record for {customer_id}"

model_with_tools = llm.bind_tools([lookup_customer])
result = model_with_tools.invoke("Look up customer u_123")
print(result.tool_calls)

For frontend and full-stack projects using @ai-sdk/openai instead of LangChain, read how to use @ai-sdk/openai with Claude, Gemini and more.

Setting spend limits and tracking usage

Unbounded LangChain agent loops can quickly exhaust API balances. thelastapi includes safety controls at the platform layer to prevent unexpected overages:

  1. Daily safety cap: Accounts default to a $5 daily spending ceiling across AI and background actions. Account owners can raise this up to $1,000 via Billing or using the CLI command vibiz billing limit <dollars>.
  2. Project AI budgets: Each project defaults to a $1 daily AI limit. When reached, requests receive a 429 rate_limited response with details.reason: "project_budget". You can modify this with:
    Terminal
    vibiz ai budget 10 --project prj_123
  3. End-user attribution: You can pass an end-user identifier in your LangChain calls by appending the user field or the X-Vibiz-User HTTP header. This enables end_user_daily_budget_usd protections and segments spend logs.

Every JSON completion response returns cost_micro_usd, which represents the exact micro-dollar cost of the request. You can check current spend figures and token counts directly from your shell:

Terminal
# View usage by model over the last 7 days
vibiz usage --period 7d --by model

# View usage segmented by end user
vibiz usage --by end_user --project prj_123

Detailed pricing mechanics, balance top-ups, and auto-recharge settings are documented in Billing and credits and the AI overview.

Next steps

To integrate your LangChain pipeline with the gateway, generate a project key using vibiz add ai or configure your existing development credentials. You can inspect all currently available models and real-time token pricing at any time using vibiz ai models --all.

Frequently asked questions

Can I use LangChain memory components like ConversationBufferMemory through the gateway?

Yes. Gateway calls use the standard OpenAI chat completions endpoint, so all standard LangChain message abstractions and memory classes pass their message arrays as usual.

What happens if a specific AI model is down while my LangChain pipeline runs?

The gateway returns a 503 error status with `model_unavailable` or `provider_unavailable` instead of silently routing your prompt to a different model. Your LangChain error handling catches this standard API error so your application can retry or notify the user.

Can I route Google Gemini requests through my own Google key?

Yes. You can register your Google API key using `GEMINI_API_KEY=... vibiz ai keys set google`, which causes Gemini models (including `vibiz/balanced`) to execute against your own Google account at zero billing charge on the gateway.

How do rate limits work when calling multiple models through LangChain?

Project keys allow 120 requests and 200,000 tokens per minute by default, while account keys allow 300 requests and 400,000 tokens per minute. If you exceed limits or hit your daily budget ceiling, the gateway returns a 429 status code with a descriptive `details.reason` field.