# Keep every scrape: run history for web data calls

> Every web data call automatically saves its input, output, duration, and cost to a searchable run history available via CLI, API, and console.

**Every web data call you make with thelastapi automatically saves its input, output, cost, and duration to run history.** You do not need to configure external storage buckets, write custom database hooks, or manually archive JSON payloads. Whether you execute a scrape, crawl a domain, or pull social posts, the platform records the execution details and assigns a run identifier immediately.

This guide explains what gets stored, how to retrieve historical scraping data through the CLI and REST API, and how to prevent duplicate runs from wasting your credits.

## What gets recorded for every web data call

When you call any action in the web data suite—such as `web.scrape`, `web.crawl`, or `places.search`—the runner captures the execution context. Every run is preserved regardless of whether it was triggered from the browser console, the command line, an autonomous agent, or your production backend.

The run history stores:

- **Action name and status**: Identifies the action (such as `web.scrape`) and records whether it completed successfully or encountered an error.
- **Input parameters**: The exact payload sent to the action, with sensitive secrets excluded and long text fields trimmed.
- **Output data**: The structured JSON result up to 1 MB in size. If a massive crawl or deep page scrape exceeds this threshold, the payload is safely cut and marked with `truncated: true`.
- **Cost**: The raw cost in micro-USD (`cost_micro_usd`), allowing you to trace expenses down to the exact URL.
- **Execution metadata**: Total duration in milliseconds, timestamps, and error codes when calls fail.

Because run tracking is built into the action runtime, you can [scrape a website to markdown with one API call](https://thelastapi.dev/blog/scrape-website-to-markdown-api) without needing an extra logging database to see what was extracted yesterday.

## Inspecting run history across your tools

You can inspect previous runs in four ways depending on where you are working:

| Interface | Command or Route | Primary Purpose |
| --- | --- | --- |
| CLI summary | `vibiz runs [--action <name>] [--since <window>]` | Review recent runs and costs from your terminal |
| CLI single run | `vibiz runs <run_id> [--json]` | Inspect the full input and output of a specific execution |
| REST API list | `GET /v1/actions/runs` | Programmatically filter runs across projects or actions |
| REST API item | `GET /v1/actions/runs/:id` | Fetch the stored payload (up to 1 MB) for an application |
| Web Console | `/web-data` (or the project **Web data** tab) | Visual history, filtering, and debugging under the runner form |
| MCP Agent Tool | `vibiz_action_runs` | Allow AI agents to check prior results before scraping again |

To view recent web actions from your local development environment, run `vibiz runs`:

```bash
# List all actions run in the last 24 hours
vibiz runs --since 1d

# Filter down to scrape jobs only
vibiz runs --action web.scrape

# Scope the list to a specific project
vibiz runs --project prj_123
```

When you need the full output of a past extraction, pass the run identifier:

```bash
vibiz runs arun_987654321 --json
```

This prints the complete stored JSON object, letting you pipe past markdown, lead lists from when you [find businesses on Google Maps with an API](https://thelastapi.dev/blog/google-maps-businesses-api-leads), or social records directly into local files or analysis tools like `jq`.

## Fetching run history through the HTTP API

Your application can retrieve past outputs without having to re-scrape websites or re-crawl directories. Every time you trigger an action over HTTP, the response includes a `run_id` and a direct `console_url`:

```bash
curl https://thelastapi.dev/v1/actions/web.scrape \
  -H "Authorization: Bearer $VIBIZ_API_KEY" \
  -H "content-type: application/json" \
  -d '{"input":{"url":"https://example.com","mainContentOnly":true}}'
```

The response provides immediate metadata:

```json
{
  "object": "action_result",
  "action": "web.scrape",
  "output": {
    "url": "https://example.com",
    "statusCode": 200,
    "title": "Example Domain",
    "markdown": "This domain is for use in illustrative examples..."
  },
  "cost_micro_usd": 800,
  "run_id": "arun_123456789",
  "console_url": "https://thelastapi.dev/web-data?action=web.scrape&run=arun_123456789"
}
```

### Listing past runs

To view past executions programmatically, call `GET /v1/actions/runs`. This endpoint returns runs in reverse chronological order (newest first) without full output payloads to keep responses fast and lightweight:

```bash
curl "https://thelastapi.dev/v1/actions/runs?action=web.scrape&limit=25" \
  -H "Authorization: Bearer $VIBIZ_API_KEY"
```

You can pass several optional query parameters to refine your query:

- `action`: Filter by a specific action, such as `web.scrape`, `web.crawl`, or `ads.library`.
- `project`: Pass a project id, or `none` to query runs not associated with any project.
- `since`: Filter by an ISO 8601 timestamp.
- `cursor`: A cursor string returned by previous queries for pagination.
- `limit`: The number of runs to return (defaults to 25; maximum is 100).

### Fetching a single run and output

Once you have a `run_id`, fetch the full record including its original output payload:

```bash
curl https://thelastapi.dev/v1/actions/runs/arun_123456789 \
  -H "Authorization: Bearer $VIBIZ_API_KEY"
```

If the scrape originally extracted a large payload, verify whether the `truncated` boolean is true. Truncated records represent responses that hit the 1 MB storage limit. For crawls, full outputs contain individual page markdown blocks inside the `pages` array.

## How AI agents reuse past runs to cut costs

Autonomous agents running loops can quickly burn through API quotas if they repeatedly scrape the same URLs. With MCP tools enabled, your agents can check existing runs before issuing new web extraction commands.

When using tools like Claude Code, Cursor, or Codex, the agent has access to `vibiz_action_runs`. An agent workflow typically looks like this:

1. **Check existing records**: The agent calls `vibiz_action_runs { action: "social.posts" }` to review past scrapes when analyzing [Instagram and TikTok profile and hashtag data with an API](https://thelastapi.dev/blog/instagram-tiktok-profile-data-api).
2. **Retrieve prior results**: If the relevant target was queried recently, the agent calls `vibiz_action_runs { id: "arun_..." }` and consumes the cached output.
3. **Execute only when necessary**: If the data is absent or outdated, the agent calls `vibiz_run_action` with `web.scrape` or `web.crawl`.

Additionally, the `web.scrape` action includes a `fresh` input parameter that defaults to `false`. When `fresh` is false, the service can reuse a copy fetched within the last two days rather than requesting the external web page again.

## Screenshots and assets in project storage

Run history tracks textual outputs directly in the run payload, but binary assets like screenshots receive dedicated file handling.

When you run `web.screenshot`:

```bash
vibiz run web.screenshot --input '{"url":"https://example.com","fullPage":true}'
```

The action output contains `image.url`. If you run the action within a project context (using a project key, passing `project_id`, or working within a project folder in the CLI), the screenshot image is permanently written to the project's Files storage inside the `web-data` folder. The resulting `image.url` becomes a permanent link that remains valid indefinitely.

If you execute `web.screenshot` without a project context, the image link expires after 24 hours. The run metadata itself remains in your history, but the binary file link will stop serving.

## Web data pricing and limits

Run history does not cost extra; it is included with action execution. Every action execution reports its raw expense in `cost_micro_usd`. The account is billed that cost plus a 30% margin drawn from your available credits.

| Action Category | Action Name | Typical Base Cost |
| --- | --- | --- |
| Page Scrapes | `web.scrape` | Fraction of a cent |
| Site Crawls | `web.crawl` | ~0.1¢ per page |
| Web Searches | `web.search` | Fraction of a cent |
| Ad Library | `ads.library` | ~0.5¢ per call |
| Map Places | `places.search` | ~0.5¢ per call |
| Keyword Metrics | `seo.keywords` | ~8¢ per batch (up to 1,000 keywords) |

All web data actions share the account's daily spend ceiling ($5 per day by default), which can be adjusted by the account owner. If an action exceeds execution bounds (between 60 and 150 seconds depending on the action), it fails with `provider_unavailable` and `details.reason: "timeout"`. Any failed action that began executing still records a run entry, logging the error and the partial cost used.

To learn more about input flags, schemas, and scraping limits, see the [Web data: scrape, search, SEO, ads library documentation](https://thelastapi.dev/docs/actions).

To explore run history in your own account, list your latest runs with `vibiz runs` or create your account at [thelastapi.dev/start](https://thelastapi.dev/start).

## Frequently asked questions

### How long are web data scrape outputs retained in run history?

Run history records are permanently retained in your console and API history. Text outputs up to 1 MB are stored with the run, while standalone screenshots generated without a project expire after 24 hours.

### How can I tell if a large crawl or scrape output was cut short?

When a scrape or crawl result exceeds the 1 MB storage limit, the output payload sets the `truncated` field to `true`.

### Does retrieving a past run from history cost additional credits?

No. Fetching past runs through `GET /v1/actions/runs`, `GET /v1/actions/runs/:id`, or the `vibiz runs` CLI command is free and does not charge against your daily spend ceiling.

---

Published 2026-11-15 · https://thelastapi.dev/blog/web-data-run-history
