Skip to content

The AI step

LLM batch API for scraped pages: last night's JSONL, matched back

Send a night's collected pages to an LLM batch API, match every answer back to its URL by custom_id, and route errored and expired requests into the next batch.

By Caitlin Sanders, AgentScraper team · Published

This article was drafted with AI assistance from the sources listed below and edited and published by the AgentScraper team. Disclosure: the AgentScraper API, which this article mentions, is this site's own product.

From last night's pages to matched answers
  1. 1CollectThe scheduled fetch writes one JSONL line per URL, success or error.
  2. 2FilterOnly lines with status success and Markdown go to the model.
  3. 3Keycustom_id is a hash of the URL: 1 to 64 characters of a-z, A-Z, 0-9, _ and -.
  4. 4SubmitOne batch per night; results can take up to 24 hours.
  5. 5JoinMatch results by custom_id, never by line order.
  6. 6Carry overErrored and expired requests go into the next batch.

Your scheduled job collects a few hundred public pages every night. In the morning you want a label or a short summary per page. Nobody reads those answers at 3 a.m., so paying for an instant model call per page makes little sense. Both large model providers sell a slower lane for exactly this: a batch API at half the usual price, with results within a day.

The catch is the join. Batch results come back in any order, and the ID field has rules a URL does not meet. This walkthrough wires a fetch job's JSONL output to a batch and back, with every URL accounted for.

What the two batch APIs give you

These are the numbers from the providers' docs, checked on the run date of this post.

Anthropic Message BatchesOpenAI Batch
Price50 percent of standard prices50% lower than synchronous
WindowExpires if not done within 24 hours; most batches finish in under 1 hourcompletion_window must be 24h
Size cap100,000 requests or 256 MB50,000 requests or a 200 MB input file
InputA requests array, each with custom_id and paramsA .jsonl file, each line with custom_id, method, url, body
Order of resultsAny orderMay differ from the input order
Results kept29 days after creationOutput and error files expire after 30 days by default

Both match results to requests through custom_id. Both require it to be unique. The Anthropic API reference adds a pattern: ^[a-zA-Z0-9_-]{1,64}$. A URL fails that pattern on its first colon.

Step 1: Start from the fetch job's JSONL

The batch scripts in the Automate requests docs write one line per URL to results.jsonl, success or not. A line looks like this:

{"url": "https://example.com/changelog", "status": "success", "error": null, "fetched_at": "2026-11-12T03:00:12Z", "markdown": "# Changelog\n..."}
{"url": "https://example.com/old-page", "status": "error", "error": "CRAWL_NOT_FOUND", "fetched_at": "2026-11-12T03:00:13Z", "markdown": null}

Keep the fetch and the model in two steps, as in the AI scraper workflow guide. The fetch run ends when the file is written. The model step reads it later.

What you see: a file with one line per input URL, and errors listed next to successes.

Step 2: Build the batch input with a hashed ID

Filter to status: success. Hash each URL into a short ID. Write the requests and a lookup file in one pass.

import hashlib, json, sys

MODEL = "claude-haiku-4-5-20251001"
PROMPT = "Classify this page as relevant or not_relevant to API changes. Answer with one word.\n\n<page>\n{md}\n</page>"

def page_id(url: str) -> str:
    return "p_" + hashlib.sha256(url.encode()).hexdigest()[:40]  # 42 chars, fits ^[a-zA-Z0-9_-]{1,64}$

seen = set()
with open(sys.argv[1]) as src, open("batch-requests.jsonl", "w") as req, open("batch-ids.jsonl", "w") as ids:
    for line in src:
        row = json.loads(line)
        if row["status"] != "success" or not row["markdown"]:
            continue
        cid = page_id(row["url"])
        if cid in seen:
            continue
        seen.add(cid)
        req.write(json.dumps({"custom_id": cid, "params": {
            "model": MODEL, "max_tokens": 16,
            "messages": [{"role": "user", "content": PROMPT.format(md=row["markdown"][:8000])}]}}) + "\n")
        ids.write(json.dumps({"custom_id": cid, "url": row["url"], "fetched_at": row["fetched_at"]}) + "\n")
print(f"{len(seen)} requests")

The seen check matters. Two input lines with the same URL produce the same hash, and both providers require unique IDs. The [:8000] cap keeps one long page from dominating the token count.

What you see: batch-requests.jsonl with one request per fetched page, batch-ids.jsonl with the URL and fetch time for each ID, and a count.

For OpenAI, write each line as {"custom_id": ..., "method": "POST", "url": "<endpoint>", "body": {...}} instead, upload the file with purpose batch, and create the batch with completion_window set to 24h. One input file can target only one model.

Step 3: Submit once, collect in the morning

import anthropic, json

client = anthropic.Anthropic()
requests = [json.loads(l) for l in open("batch-requests.jsonl")]
batch = client.messages.batches.create(requests=requests)
print(batch.id)

Store the batch ID next to the run. In the morning, check that processing_status is ended before you read results. A batch cannot be changed once submitted; to fix it, cancel it and submit a new one.

What you see: an ID starting with msgbatch_, and later a status of ended.

Step 4: Join by custom_id, not by line

import anthropic, json, sys

client = anthropic.Anthropic()
lookup = {r["custom_id"]: r for r in map(json.loads, open("batch-ids.jsonl"))}
done = set()
with open("answers.jsonl", "w") as out, open("carry-over.jsonl", "w") as carry:
    for res in client.messages.batches.results(sys.argv[1]):
        page = lookup[res.custom_id]
        done.add(res.custom_id)
        kind = res.result.type
        answer = res.result.message.content[0].text if kind == "succeeded" else None
        out.write(json.dumps({**page, "model_status": kind, "answer": answer}) + "\n")
        if kind in ("errored", "expired"):
            carry.write(json.dumps(page) + "\n")
missing = set(lookup) - done
print(f"{len(done)} results, {len(missing)} missing")

Every answer line keeps fetched_at from the fetch run. The answer describes the page as it was at that time, not when the model read it. Keep both fields; they are two different clocks.

What you see: answers.jsonl with one line per page, carry-over.jsonl with what goes into the next batch, and 0 missing.

Model-side statuses

The fetch already has its own status per URL. The batch adds a second one. Route on it the same way.

ResultWhere it showsWhat to do
succeededAnthropic resultsStore the answer
errored with invalid_request_errorAnthropic resultsFix the request body; a resend as-is fails again
errored, other typesAnthropic resultsResend in the next batch
expiredAnthropic resultsResend in the next batch
canceledAnthropic resultsResend if you still need it
Error lineOpenAI file at error_file_idMatch by custom_id, then resend or fix
batch_expiredOpenAI error fileResend in the next batch

The Claude docs state that one failed request does not affect the others in the batch. OpenAI keeps the responses that finished before an expiry in the output file and charges for their tokens.

Two cost lines

The fetch and the model bill apart. On the fetch side: "Failed fetches cost 0 credits." Model tokens are billed by your model provider, at batch rates when you use a batch. That is one more reason to keep error lines out of the batch: a page that failed to fetch is free on the fetch side and pure waste on the model side.

Store the token usage per answer next to the URL. When the monthly total moves, you can tell which side moved.

What this means for your workflow

  • Write the fetch output first, one JSONL line per URL. The model step reads it later.
  • Send only status: success lines with Markdown, deduped.
  • Use a URL hash as custom_id; keep a lookup file with the URL and fetched_at.
  • Join results by custom_id. Count missing IDs on every run.
  • Put errored and expired requests into the next night's batch, after fixing invalid ones.
  • Download results the same day; they are not kept forever.

For the fetch half's own retry rules, see the retry policy for batch fetches.

Frequently asked questions

Do LLM batch APIs return results in the same order as the requests?

No. Both the Anthropic Message Batches API and the OpenAI Batch API say results can come back in any order. Match each result to its request with the custom_id field.

Can I use the page URL as the custom_id?

Not on the Anthropic Message Batches API. Its custom_id must be 1 to 64 characters of letters, digits, underscores and hyphens, and must be unique within the batch. A URL has slashes and colons and is often longer, so use a short hash of the normalized URL and keep a lookup file.

How long does a batch take?

Both providers give a 24-hour window. The Claude docs say most batches finish in less than 1 hour; OpenAI says batches often complete sooner than 24 hours. Requests that are not done when the window closes come back as expired.

Should failed page fetches go into the batch?

No. Send only pages whose fetch status is success and that have Markdown. A failed fetch has no page text, so a model call on it costs tokens and returns nothing useful.

Sources

  1. Batch processing (Claude docs) (Documentation)
  2. Create a Message Batch (Claude API reference) (Documentation)
  3. Batch API (OpenAI docs) (Documentation)