# LangChain with a multi-model OpenAI-compatible gateway

> Connect LangChain to multiple AI models using an OpenAI-compatible gateway with a single base URL, one API key, and zero model swapping.

**You can connect LangChain to multiple AI labs by pointing LangChain's `ChatOpenAI` class at an OpenAI-compatible gateway URL and passing your gateway API key.** thelastapi provides an OpenAI-compatible endpoint at `/v1/ai` that lets you call models from OpenAI, Anthropic, Google, DeepSeek, Moonshot AI, and Meta without installing separate lab libraries. You keep your existing LangChain chains, prompts, and tool configurations while switching between models simply by updating the model name string.

This guide walks through configuring LangChain in Python and TypeScript, routing to different lab models through one interface, and monitoring costs and rate limits.

## How a multi-model gateway works with LangChain

LangChain includes native wrappers for many individual model providers, but managing separate packages creates maintenance friction. Each provider requires its own client initialization, credential handling, and version management. If you want to benchmark an Anthropic model against a Google model, you usually need separate dependencies, separate account dashboards, and separate billing cards.

An OpenAI-compatible gateway changes this workflow. Because LangChain's `ChatOpenAI` integration expects the standard OpenAI Chat Completions REST format, you can route requests through any gateway that implements that contract. 

When your LangChain code executes a prompt, the gateway receives the request at `/v1/ai/chat/completions`, runs the exact model you specified, and streams back the response in standard OpenAI JSON or SSE chunks. thelastapi never silently swaps models if one provider encounters downtime; if a model is unreachable, the API returns a `503` with a `model_unavailable` or `provider_unavailable` error code. To understand the underlying transport architecture, see our guide on [how a unified AI API works](https://thelastapi.dev/blog/one-api-key-gpt-claude-gemini).

## Setting up your credentials and environment

To use the gateway with LangChain, you need a project AI key or account key. You can initialize a project using the CLI:

```bash
vibiz init "LangChain Assistant" --resources ai
vibiz dev
```

The `vibiz dev` command provisions your local development environment and populates your local environment file with the necessary connection details. It provides `VIBIZ_AI_BASE_URL` and `VIBIZ_AI_API_KEY`, alongside the standard `OPENAI_BASE_URL` and `OPENAI_API_KEY` aliases.

The standard endpoint for AI calls is:
- **Base URL:** `https://thelastapi.dev/v1/ai`
- **Key Format:** `vz_ai_...` (for a project AI key) or `vz_sk_...` (for an account key)

If you are using `@vibiz/app` within an existing application, the SDK exposes your credentials directly via `vibiz.ai`. You can read more about managing access tokens and security boundaries in the [Keys and environments](https://thelastapi.dev/docs/keys) documentation.

## Configuring LangChain to call multiple models

Because the gateway supports standard OpenAI request structures, you instantiate LangChain's `ChatOpenAI` client directly. You do not need to install provider-specific integrations like `@langchain/anthropic` or `langchain-google-genai`.

### Python setup

Install the standard LangChain OpenAI package:

```bash
pip install langchain-openai
```

Point the client to the gateway endpoint:

```python
import os
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage

# Initialize the client pointing to the gateway
llm = ChatOpenAI(
    base_url=os.environ.get("VIBIZ_AI_BASE_URL", "https://thelastapi.dev/v1/ai"),
    api_key=os.environ.get("VIBIZ_AI_API_KEY"),
    model="vibiz/smart"
)

response = llm.invoke([
    HumanMessage(content="Explain distributed transactions in three sentences.")
])
print(response.content)
```

### TypeScript setup

Install the JavaScript package:

```bash
npm install @langchain/openai
```

Instantiate `ChatOpenAI` using the `configuration` parameter:

```ts
import { ChatOpenAI } from '@langchain/openai';
import { HumanMessage } from '@langchain/core/messages';

const model = new ChatOpenAI({
  modelName: 'vibiz/balanced',
  configuration: {
    baseURL: process.env.VIBIZ_AI_BASE_URL || 'https://thelastapi.dev/v1/ai',
    apiKey: process.env.VIBIZ_AI_API_KEY,
  },
});

const res = await model.invoke([
  new HumanMessage('Generate three distinct naming ideas for an uptime monitoring product.'),
]);
console.log(res.content);
```

For teams considering whether to connect directly to each provider or standardize on an adapter layer, read our breakdown on [using OpenAI directly vs a model gateway](https://thelastapi.dev/blog/openai-direct-vs-model-gateway).

## Choosing between stable tiers and direct model IDs

thelastapi provides two ways to address models in your LangChain components: functional model tiers and explicit lab identifiers.

### 1. Functional tiers

Functional tiers are stable model names mapped to specific lab versions. When a lab releases an update, the underlying model is upgraded, but your LangChain configuration string remains the same. Every API response includes a `served_model` field so you always know which exact checkpoint handled the generation.

| Model Tier | Underlying Model | Primary Use Case | Price per 1M tokens (In / Out) |
| --- | --- | --- | --- |
| `vibiz/fast` | GPT-6 Luna (OpenAI) | Chat, tool calling, JSON parsing | $0.10 / $0.50 |
| `vibiz/lite` | DeepSeek V4.1 Flash | Cost-efficient generation, 1M context | $0.14 / $0.42 |
| `vibiz/smart-mini` | Claude Haiku 5.5 (Anthropic) | Fast coding and technical writing | $0.10 / $0.50 |
| `vibiz/balanced` | Gemini 3.8 Flash (Google) | Balanced quality, 1M context | $0.825 / $4.125 |
| `vibiz/reasoning` | GPT-6.1 Terra (OpenAI) | Multi-step logic, math, planning | $1.00 / $5.00 |
| `vibiz/smart` | Claude Sonnet 5.5 (Anthropic) | Complex coding, long chains | $2.00 / $10.00 |
| `vibiz/pro` | Claude Opus 5.5 (Anthropic) | Deep reasoning and evaluation | $4.00 / $20.00 |

### 2. Direct model IDs

You can also pass exact provider IDs directly to the `model` parameter. Over 400 models from OpenAI, Anthropic, Google, DeepSeek, Moonshot AI, Mistral, Qwen, and Meta are available. For example:

```python
# Call Claude Opus directly
opus = ChatOpenAI(
    base_url="https://thelastapi.dev/v1/ai",
    api_key=os.environ["VIBIZ_AI_API_KEY"],
    model="anthropic/claude-opus-5.5"
)

# Call Moonshot Kimi directly
kimi = ChatOpenAI(
    base_url="https://thelastapi.dev/v1/ai",
    api_key=os.environ["VIBIZ_AI_API_KEY"],
    model="moonshotai/kimi-k3.1-pro"
)
```

If you specify an invalid or mistyped model string, the gateway returns a `400 invalid_request` with `param: "model"`. It will never default to a fallback model without your consent.

## Managing vector embeddings and tool calling

LangChain workflows often require vector embeddings and function execution alongside text generation.

### Vector embeddings

You can use standard LangChain embedding classes pointing to `/v1/ai/embeddings`:

```python
from langchain_openai import OpenAIEmbeddings

embeddings = OpenAIEmbeddings(
    base_url="https://thelastapi.dev/v1/ai",
    api_key=os.environ["VIBIZ_AI_API_KEY"],
    model="vibiz/embed"  # text-embedding-3-small, 1536 dims, $0.02 / 1M tokens
)

vector = embeddings.embed_query("Query string to vectorize")
```

For higher dimensionality tasks, `vibiz/embed-large` maps to `text-embedding-3-large` (3,072 dimensions) at $0.13 per million tokens. The embedding batch limit accepts up to 2,048 inputs per request.

### Tool calling

The gateway supports OpenAI-formatted function calling across compatible models, including `vibiz/fast` and `vibiz/smart`:

```python
from langchain_core.tools import tool

@tool
def lookup_customer(customer_id: str) -> str:
    """Retrieves account details for a customer."""
    return f"Active customer record for {customer_id}"

model_with_tools = llm.bind_tools([lookup_customer])
result = model_with_tools.invoke("Look up customer u_123")
print(result.tool_calls)
```

For frontend and full-stack projects using `@ai-sdk/openai` instead of LangChain, read [how to use @ai-sdk/openai with Claude, Gemini and more](https://thelastapi.dev/blog/ai-sdk-openai-provider-any-model).

## Setting spend limits and tracking usage

Unbounded LangChain agent loops can quickly exhaust API balances. thelastapi includes safety controls at the platform layer to prevent unexpected overages:

1. **Daily safety cap:** Accounts default to a $5 daily spending ceiling across AI and background actions. Account owners can raise this up to $1,000 via Billing or using the CLI command `vibiz billing limit <dollars>`.
2. **Project AI budgets:** Each project defaults to a $1 daily AI limit. When reached, requests receive a `429 rate_limited` response with `details.reason: "project_budget"`. You can modify this with:
   ```bash
   vibiz ai budget 10 --project prj_123
   ```
3. **End-user attribution:** You can pass an end-user identifier in your LangChain calls by appending the `user` field or the `X-Vibiz-User` HTTP header. This enables `end_user_daily_budget_usd` protections and segments spend logs.

Every JSON completion response returns `cost_micro_usd`, which represents the exact micro-dollar cost of the request. You can check current spend figures and token counts directly from your shell:

```bash
# View usage by model over the last 7 days
vibiz usage --period 7d --by model

# View usage segmented by end user
vibiz usage --by end_user --project prj_123
```

Detailed pricing mechanics, balance top-ups, and auto-recharge settings are documented in [Billing and credits](https://thelastapi.dev/docs/billing) and the [AI overview](https://thelastapi.dev/docs/ai).

## Next steps

To integrate your LangChain pipeline with the gateway, generate a project key using `vibiz add ai` or configure your existing development credentials. You can inspect all currently available models and real-time token pricing at any time using `vibiz ai models --all`.

## Frequently asked questions

### Can I use LangChain memory components like ConversationBufferMemory through the gateway?

Yes. Gateway calls use the standard OpenAI chat completions endpoint, so all standard LangChain message abstractions and memory classes pass their message arrays as usual.

### What happens if a specific AI model is down while my LangChain pipeline runs?

The gateway returns a 503 error status with `model_unavailable` or `provider_unavailable` instead of silently routing your prompt to a different model. Your LangChain error handling catches this standard API error so your application can retry or notify the user.

### Can I route Google Gemini requests through my own Google key?

Yes. You can register your Google API key using `GEMINI_API_KEY=... vibiz ai keys set google`, which causes Gemini models (including `vibiz/balanced`) to execute against your own Google account at zero billing charge on the gateway.

### How do rate limits work when calling multiple models through LangChain?

Project keys allow 120 requests and 200,000 tokens per minute by default, while account keys allow 300 requests and 400,000 tokens per minute. If you exceed limits or hit your daily budget ceiling, the gateway returns a 429 status code with a descriptive `details.reason` field.

---

Published 2026-11-15 · https://thelastapi.dev/blog/langchain-multi-model-gateway
