thelastapi
Get started free

Script, voice and video: an automated short-video pipeline with one API

Build an automated short-video pipeline generating scripts, voiceovers, and 9:16 clips using one API key, one SDK, and permanent file storage.

On this page 8 sections

You can build an end-to-end pipeline that writes scripts, generates voiceover audio, and renders vertical video clips using a single API key. With thelastapi, you call language models like Claude and GPT for scripts, text-to-speech for voice tracks, and video models like vibiz/video for 9:16 clips under one base URL at /v1/ai. Every generated asset is automatically stored in your project's permanent file storage without managing separate media buckets.

Automating short-form video production typically requires duct-taping four different vendors together: one provider for script generation, a specialized voice platform for audio, a separate compute service for video rendering, and cloud storage to host the results. That approach leaves you managing multiple billing dashboards, rotating four sets of API tokens, and writing glue code to pass URLs back and forth. A unified gateway handles every stage through the standard OpenAI client shape you already know.

Here is how to design, code, and deploy an automated short-video pipeline from prompt to published asset.

How the single-key video pipeline works

The entire pipeline executes in three sequential stages through the AI API, backed by automatic asset persistence through Files:

  1. Script generation: A chat model writes a concise voiceover script and visual scene prompts formatted as structured JSON.
  2. Voiceover production: A text-to-speech call converts the voiceover script into an MP3 audio track using one of nine natural voices.
  3. Video clip generation: A video job renders a 6-second vertical (9:16) video clip at 720p with sound, driven either by a scene prompt or by an initial image still.

Because every media call is tied to your project, the resulting audio and video files receive permanent URLs immediately. You do not need to download the binary stream to your server and re-upload it to external object storage.

Text
┌────────────────────────────────────────────────────────┐
│                      thelastapi                        │
│                                                        │
│  1. Script          2. Voiceover        3. Video       │
│  vibiz/smart    →   vibiz/voice     →   vibiz/video    │
│  (JSON prompt)      (MP3 audio)         (9:16 720p)    │
│                           │                  │         │
│                           ▼                  ▼         │
│                 Project Files (Permanent URLs)        │
└────────────────────────────────────────────────────────┘

Step 1: Generate the script and scene prompts

Short-form videos on platforms like YouTube Shorts, Instagram Reels, and TikTok require punchy copy and tight visual directions. You can use Anthropic's Claude Sonnet 5.5 through the stable model alias vibiz/smart, or OpenAI's GPT-6 Luna via vibiz/fast, requesting strict JSON output.

Install the official OpenAI SDK and configure your project keys:

Terminal
vibiz init "Shorts Engine" --resources ai
vibiz dev

Your .env.development.local file receives VIBIZ_AI_BASE_URL and VIBIZ_AI_API_KEY. The standard OpenAI SDK works without code changes:

TypeScript
import OpenAI from 'openai';

const openai = new OpenAI({
  baseURL: process.env.VIBIZ_AI_BASE_URL,
  apiKey: process.env.VIBIZ_AI_API_KEY,
});

interface VideoScript {
  title: string;
  voiceover: string;
  visualPrompt: string;
}

async function generateScript(topic: string): Promise<VideoScript> {
  const response = await openai.chat.completions.create({
    model: 'vibiz/smart',
    messages: [
      {
        role: 'system',
        content:
          'You write viral 6-second short video concepts. Return JSON with title, voiceover (under 150 characters), and visualPrompt (detailed scene description).',
      },
      {
        role: 'user',
        content: `Create a video concept about: ${topic}`,
      },
    ],
    response_format: { type: 'json_object' },
  });

  const content = response.choices[0]?.message.content ?? '{}';
  return JSON.parse(content) as VideoScript;
}

The cost_micro_usd field in the response tells you the exact cost of the generation down to the microdollar.

Step 2: Convert the script to spoken audio

Next, take the voiceover text and send it to the audio speech endpoint. The model vibiz/voice costs $0.04 per 1,000 characters and supports nine built-in voices: alloy, ash, coral, echo, fable, nova, onyx, sage, and shimmer.

When you make a speech request within a project, the API returns the streamed audio and includes two headers: X-Vibiz-File-Id and X-Vibiz-File-Url. The audio is permanently stored in your project's Files drive as soon as the stream finishes:

TypeScript
async function generateVoiceover(scriptText: string): Promise<string> {
  const speechResponse = await fetch(`${process.env.VIBIZ_AI_BASE_URL}/audio/speech`, {
    method: 'POST',
    headers: {
      Authorization: `Bearer ${process.env.VIBIZ_AI_API_KEY}`,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      model: 'vibiz/voice',
      input: scriptText,
      voice: 'onyx',
      response_format: 'mp3',
    }),
  });

  if (!speechResponse.ok) {
    throw new Error(`Speech creation failed: ${speechResponse.statusText}`);
  }

  // The permanent URL to the stored audio file
  const fileUrl = speechResponse.headers.get('x-vibiz-file-url');
  if (!fileUrl) {
    throw new Error('Permanent file URL was not returned.');
  }

  return fileUrl;
}

If you need fast turnaround in multilingual settings, vibiz/voice-lite costs $0.04 per 1,000 characters across 32 languages, while vibiz/voice-pro ($0.08 per 1,000 characters) provides extra expressiveness. Learn more about media models in our overview of one API for image, video and voice AI models.

Step 3: Render vertical video clips

Video generation takes longer than chat completion, so it runs as an asynchronous job via POST /v1/ai/jobs. The default video model vibiz/video runs LTX-2.5 Fast by Lightricks, producing 6-second 720p clips with sound for $0.54 per video.

Set aspect_ratio to 9:16 for vertical content. For more details on format configurations, see our guide on a vertical 9:16 video API for Shorts, Reels and TikTok.

TypeScript
interface JobResponse {
  id: string;
  status: 'pending' | 'processing' | 'completed' | 'failed';
  output?: {
    data: Array<{
      url: string;
      file_id: string;
      file_url: string;
    }>;
  };
  error?: { message: string };
}

async function renderVideoClip(prompt: string): Promise<string> {
  // 1. Submit the generation job
  const jobCreate = await fetch(`${process.env.VIBIZ_AI_BASE_URL}/jobs`, {
    method: 'POST',
    headers: {
      Authorization: `Bearer ${process.env.VIBIZ_AI_API_KEY}`,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      type: 'video',
      model: 'vibiz/video',
      input: {
        prompt,
        aspect_ratio: '9:16',
      },
    }),
  });

  const job = (await jobCreate.json()) as JobResponse;
  const jobId = job.id;

  // 2. Poll until the job completes
  while (true) {
    await new Promise((resolve) => setTimeout(resolve, 3000));

    const pollRes = await fetch(`${process.env.VIBIZ_AI_BASE_URL}/jobs/${jobId}`, {
      headers: {
        Authorization: `Bearer ${process.env.VIBIZ_AI_API_KEY}`,
      },
    });

    const current = (await pollRes.json()) as JobResponse;

    if (current.status === 'completed' && current.output?.data?.[0]) {
      // The video URL is permanent and points directly to project Files
      return current.output.data[0].file_url || current.output.data[0].url;
    }

    if (current.status === 'failed') {
      throw new Error(`Video job failed: ${current.error?.message}`);
    }
  }
}

You can also pass image_url inside input to animate a pre-existing still image rather than generating purely from text. If your pipeline creates visual mockups or marketing visuals first, read how to generate product photos for an online store with an API.

Running the pipeline from CLI or AI coding agents

You can run this entire sequence manually from your terminal using the vibiz CLI. This is helpful for debugging prompts or testing voice delivery before pushing changes to your pipeline script:

Terminal
# 1. Script generation
vibiz ai "Write a 5-second punchy teaser about deep ocean exploration" --model vibiz/smart

# 2. Voiceover creation
vibiz say "More people have walked on the moon than visited the Mariana Trench." -o ocean_voice.mp3 --voice onyx

# 3. Video rendering (9:16 aspect ratio)
vibiz video "Deep underwater camera moving towards glowing hydrothermal vents, 4k" -o ocean_clip.mp4 --aspect 9:16

Because the CLI and endpoints expose the standard tool schema, AI tools like Cursor and Claude Code can run these workflows directly during development sessions. Learn more about running these tasks directly from an interactive assistant in generate images and video from Claude Code.

Pipeline cost breakdown

Running an automated media generation pipeline requires predictable costs. The table below lists standard list prices for the models used across script, audio, and video generation:

Pipeline StageModelModalityOutput SpecUnit Price
Scripting (Default)vibiz/fastText / JSONGPT-6 Luna$0.10 in / $0.50 out (per 1M tokens)
Scripting (Advanced)vibiz/smartText / JSONClaude Sonnet 5.5$2.00 in / $10.00 out (per 1M tokens)
Voiceovervibiz/voiceText-to-Speech9 voices, MP3$0.04 per 1,000 characters
Voiceover (Fast)vibiz/voice-liteText-to-Speech32 languages$0.04 per 1,000 characters
Video Generationvibiz/videoText / Image to Video6s clip, 720p, sound$0.54 per video
File StorageFilesMedia storagePermanent URLsFree during beta

A standard 6-second video asset using vibiz/fast for copy, a 120-character voice track via vibiz/voice, and a vertical clip via vibiz/video costs approximately $0.55 per generated asset.

Managing rate limits and project budgets

Automated generation jobs can spend credits quickly if an external queue loops or encounters an error. thelastapi enforces guardrails at both the account and project levels:

  • Daily spend limit: Accounts default to a $5 daily ceiling across all AI calls and actions. Account owners can increase this ceiling in Billing settings.
  • Project budget: Projects start with a default $1 daily budget. You can inspect or raise this limit using the CLI:
    Terminal
    vibiz ai budget 25 --project prj_123
  • Concurrency & rate caps: Project keys allow up to 120 requests and 200,000 tokens per minute, with up to 25 concurrent requests in flight. Account keys allow 300 requests and 400,000 tokens per minute.
  • Budget alerts: Owners receive automated email alerts whenever usage reaches 80% and 100% of the project budget or account spend limit.

If a batch worker hits a budget ceiling, the API responds with HTTP status 429 and details.reason: "project_budget", avoiding unexpected overages.

To start assembling your media pipeline, create a project with vibiz init, inspect model prices with vibiz ai models, and consult the AI documentation for full schema details.

Frequently asked questions

Where are the generated video and audio files hosted?

Every audio, image, and video file created within a project is automatically stored in your project's Files drive. The response contains a permanent link formatted as `https://thelastapi.dev/f/<key>/<name>`, which does not expire.

Can I use another lab's video model instead of LTX-2.5 Fast?

Yes. You can query alternative models such as Kling, Seedance, or Veo using `GET /v1/ai/media-models` or `vibiz media-models` and run them by id on `POST /v1/ai/jobs` using the `model` parameter.

What aspect ratios are supported for video generation?

The video model accepts either `9:16` for vertical shorts and reels or `16:9` for horizontal videos. If you submit `1:1`, the API defaults to `16:9`.

What happens if an automated pipeline exceeds the daily AI budget?

The API returns an HTTP 429 status code with the error details reason set to `daily_spend` or `project_budget`. Calls will not silently run or bill beyond the ceiling configured by the account owner.