Claude Batch API: Cutting Async Workload Costs In Half

Last updated: November 4, 2026

Updated October 18, 2026: The code samples check out against @anthropic-ai/sdk 0.135.0, but several claims around them did not match Anthropic's batch processing docs. The Message Batches API has no completion webhook (the SDK's beta webhooks cover Managed Agents events, not batches), so the webhook section is now a polling section. Batch usage does not count against Messages API rate limits, cache hits inside a batch are best-effort (Anthropic cites 30% to 98%), server-side tools do run inside batches, and results are downloadable for 29 days. The cost example was also off: at Haiku 4.5 rates the synchronous run is about $425, not $1,275, and the other figures are recomputed below.

Official Sources#

A 50% Discount Hiding In Plain Sight#

If you have a Claude workload that doesn't need to answer in real time, you are probably overpaying by 2x. The Batch API charges half - half - the per-token rate of the synchronous API in exchange for results within 24 hours. For nightly reports, content classification, synthetic data generation, document analysis, and most agent evaluations, that trade is a no-brainer. And yet most teams I look at still run those workloads through the live API because nobody wants to refactor the request loop.

This is the version of the docs I wish I had the first time I moved a real workload to batch. We will cover the request format, the SDK code you should ship, the architecture changes async forces on your stack, and the failure modes you only learn about after a 100k-request batch silently drops 4% of its results.

We walked through a real migration in our Batch Processing for Scale: 100k Requests/Day video on YouTube. This post is the production-grade companion.

When Batch Wins, And When It Loses#

Batch is the right answer when:

• The user doesn't see the result immediately. Reports, classification jobs, ETL pipelines, eval suites.
• You can tolerate up to a 24-hour SLA. Most batches finish in well under an hour, but you should not promise less than 24h to your callers.
• Volume is non-trivial. Below ~1000 requests per batch, the operational overhead of async eats the cost win.
• The cost per request matters. At 5 cents a call, 50% off is a real number; at 0.05 cents a call, the engineering cost dwarfs the savings.

Batch is the wrong answer when:

• A user is waiting on the response. Latency is unbounded.
• The workload is spiky and unpredictable. Batches benefit from steady throughput.
• Each request depends on the last. Conversations, agent loops, and anything stateful belong on the synchronous API.

The hybrid pattern - synchronous for user-facing work, batch for everything else - is what mature deployments look like. Customer-asked-a-question goes live; nightly data labeling goes batch.

That split matters more with Claude Haiku 5.5: synchronous IDE and subagent work can use Haiku for bounded steps, while offline labeling, compaction and evaluation runs can stack Haiku pricing with the Batch API discount.

What Batch Actually Is, Mechanically#

You package up to 100,000 message-create requests (or 256 MB, whichever comes first) into a single batch. You submit it. Anthropic processes each request independently and asynchronously. You poll for completion. When done, you stream a JSONL results file and match results back to your inputs by custom_id.

Three properties to internalize:

1. Each row is an independent message-create call. You get the full request body - model, messages, tools, system, thinking, cache_control, everything. Batch is "synchronous API at half price with a delayed return," not a different API.
2. Order is not preserved. Results come back in whatever order they finished. You match on custom_id. Anyone who ignores this and zips inputs to outputs gets bitten.
3. Failures are per-request, not per-batch. Each row ends as succeeded, errored, canceled or expired, and one failed row does not affect the others. Anthropic does not bill errored, canceled or expired rows. You must scan the result file for them.

The TypeScript Code You Should Actually Ship#

Here is a minimal but production-shaped batch submission using the official Anthropic SDK. It builds requests with stable IDs, submits, polls, and reconciles results.

import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic();

type ClassifyInput = { id: string; text: string };

const SYSTEM = "Classify the input as one of: bug, feature, question, spam. Return only the label.";

function buildRequests(inputs: ClassifyInput[]) {
  return inputs.map((input) => ({
    custom_id: input.id,
    params: {
      model: "claude-haiku-4-5" as const,
      max_tokens: 16,
      system: SYSTEM,
      messages: [{ role: "user" as const, content: input.text }],
    },
  }));
}

export async function classifyBatch(inputs: ClassifyInput[]) {
  const batch = await client.messages.batches.create({
    requests: buildRequests(inputs),
  });

  let status = batch;
  while (status.processing_status !== "ended") {
    await new Promise((r) => setTimeout(r, 30_000));
    status = await client.messages.batches.retrieve(batch.id);
  }

  const results: Record<string, string> = {};
  for await (const result of await client.messages.batches.results(batch.id)) {
    if (result.result.type === "succeeded") {
      const block = result.result.message.content[0];
      results[result.custom_id] = block.type === "text" ? block.text.trim() : "";
    } else {
      results[result.custom_id] = `__error__:${result.result.type}`;
    }
  }
  return results;
}

A few non-obvious things this captures:

• custom_id is yours to define and must be unique per batch: 1 to 64 characters, letters, digits, hyphens and underscores only. We use whatever primary key the input row has in our database. This is how you reconcile results to records.
• The SDK's results() iterator streams a JSONL file. Do not try to load it into memory for large batches; iterate.
• The polling loop here is naive. In production, use a worker that wakes on a schedule. A 30-second poll is fine for prototyping; do not run it for thousands of batches in parallel.

Polling Without A Fleet Of Sleepers#

The Message Batches API does not push a notification when a batch ends; the docs poll batches.retrieve() until processing_status is ended (their example sleeps 60 seconds between checks). The @anthropic-ai/sdk beta webhooks cover Managed Agents events such as sessions and deployments, not message batches.

What fails at scale is one sleeping loop per batch. Run a single scheduled worker instead:

1. Every few minutes, list the batches you still have open (from your own jobs table, or client.messages.batches.list())
2. Retrieve each one and skip any that are not ended
3. Stream the results with batches.results() and write them to your database
4. Mark the job complete in the same transaction, so a rerun of the worker does not process the batch twice

Make step 3 idempotent on custom_id. A worker that crashes halfway will see the same batch again on its next pass.

Reconciliation: The Single Most Important Step#

The bug you don't notice in dev and absolutely will hit in prod: a 99,500-row result file for a 100,000-row batch, and you ship the answer assuming all rows came back.

The reconciliation pattern that survives:

async function reconcile(batchId: string, expectedIds: Set<string>) {
  const seen = new Set<string>();
  const errors: { id: string; reason: string }[] = [];
  const ok: { id: string; output: string }[] = [];

  for await (const r of await client.messages.batches.results(batchId)) {
    seen.add(r.custom_id);
    if (r.result.type === "succeeded") {
      const block = r.result.message.content[0];
      ok.push({ id: r.custom_id, output: block.type === "text" ? block.text : "" });
    } else {
      errors.push({ id: r.custom_id, reason: r.result.type });
    }
  }

  const missing = [...expectedIds].filter((id) => !seen.has(id));
  return { ok, errors, missing };
}

You always end up with three buckets: succeeded, errored, and missing. Each bucket needs a path. Errored rows are eligible for retry on the live API or a follow-up batch. Missing rows are the silent failure mode and almost always indicate a bug in how you constructed the batch (duplicate custom_ids, oversized rows, encoding problems).

Cost Modeling: Where The 50% Actually Lands#

The 50% discount applies to input tokens, output tokens and "any special tokens", per Anthropic's batch FAQ, and the prompt caching discount stacks on top. Thinking tokens bill as output, so they are halved too. There is no separate "batch tier" to opt into beyond using the batch endpoint.

The savings stack with caching. A nightly classification batch of 100k rows with a 4k-token shared instruction and 50 output tokens per row is 400M input and 5M output tokens. At Claude Haiku 4.5 rates ($1 input, $5 output, $0.10 cache read per million tokens; batch halves all three):

• Sync: about $425 ($400 input, $25 output)
• Sync + caching: about $65 if nearly every call reads the cached instruction
• Batch, no caching: about $212
• Batch + caching: about $36 at a 98% cache hit rate, about $159 at 30%

The batch-plus-cache figure is a range because cache hits inside a batch are best-effort. Anthropic's docs say users typically see 30% to 98%, depending on traffic patterns.

For workloads with stable instructions and high request counts, batch + cache is the cheapest configuration available on the Claude API today.

For monitoring batch spend over time and catching regressions, CodeBurn tracks per-batch tokens and per-job cost so you can spot the runaway batch before the invoice shows up.

Real-World Batch Workloads#

1. Daily content classification. Tag a few hundred thousand rows of incoming user content with categories. Haiku in batch mode runs about $1 per 100k rows for short text. We have shipped this on three different products.

2. Scheduled report generation. Generate per-customer summaries overnight. The latency floor is fine because the report lands in the inbox at 7am anyway. Sonnet in batch mode is the right model.

3. Synthetic data generation. Generating eval datasets, training sets, or persona-conditioned variants. These are big, embarrassingly parallel, and not user-facing. Perfect batch fit.

4. Eval suites. Running a 10k-prompt eval against five models takes 50,000 calls. Batch all of them, reconcile, compute metrics. Cuts both cost and wall-clock time vs. naive serial calling.

5. Document re-indexing. Re-summarizing or re-extracting from a corpus when you change your prompt. Every team I know that runs RAG eventually has a "rebuild all summaries" job; this is what batch is for.

Production Gotchas Worth Pinning To Your Wall#

1. Per-batch row limits. Up to 100,000 requests or 256MB total, whichever hits first. For larger jobs, shard into multiple batches. A simple sharding scheme: hash custom_id mod N.

2. Batches have their own rate limits. The Batches API limits HTTP requests and the number of requests waiting to be processed, and Anthropic's FAQ says batch usage does not affect your Messages API rate limits. What does happen under load is slower processing: the docs warn that high demand can mean more requests expiring at the 24-hour mark, so plan a retry path for expired rows.

3. Cache hits are best-effort. Requests in a batch run concurrently and in any order, so you cannot make the first row write the cache for the rest. Anthropic's guidance is to put identical cache_control blocks in every request, share as much prefix as possible, and keep a steady stream of requests so entries do not hit their 5-minute expiry. The SDK does nothing extra here; it sends what you put in each row.

4. Client tools cannot loop inside a batch. If a row calls one of your own tools, the result comes back with the tool call and you have to run it yourself and submit a follow-up request. Anthropic's server-side tools (code execution, MCP connectors, tool search and the rest) are different: the batch worker runs the same server-side loop as the synchronous API, and a pause_turn result means you continue the turn in a follow-up request.

5. The 24-hour SLA is a ceiling, not a target. Anthropic says most batches complete within 1 hour, and requests not processed within 24 hours expire. Median completion in our measurements is 5 to 30 minutes. Plan for 24h, but don't design dashboards that show "still processing" for the first 12 hours and panic users.

6. Results expire after 29 days; batch IDs should not. Anthropic keeps batch results downloadable for 29 days after creation. You will want to query by batch ID six months later when an auditor asks where row 47,318 of a March classification job went, so store the results and the batch ID alongside the records they generated, not in a side table that gets pruned.

Hybrid Architecture: The Endgame#

The pattern mature production stacks land on:

[user request]
    |- if interactive -> [synchronous API w/ caching]
    |- if backgroundable -> [enqueue]
                            |- [batch worker pulls every N min]
                                  |- [submits batch]
                                        |- [scheduled poller -> reconcile -> notify]

The interesting decision is the boundary. We move workloads to batch the moment we can answer "is anyone watching the spinner" with "no." Reports, daily syncs, classification, evals, summaries, embeddings (when we still need them). Anything user-facing stays synchronous, often on Haiku, often with thinking off.

The 400-Dollar Overnight Bill post-mortem walks through what happens when this boundary blurs and a "should be batchable" job ends up on the live API at full price.

Production Checklist Before You Ship#

• custom_id set to your stable record ID, unique per batch
• Reconciliation step that produces succeeded / errored / missing buckets
• One scheduled poller for all open batches, not a sleeping loop per batch
• Idempotent result writes, keyed on custom_id
• Retry path for expired rows
• Batch ID stored alongside generated records
• Per-batch token cost logged and alertable
• Sharding strategy for jobs over 100k rows
• Retry strategy for errored rows (live API or follow-up batch)
• Monitoring on batch completion latency, alert on stragglers > 12h
• Documented latency SLA to upstream callers (24h, not 1h)

The Batch API is the cheapest legitimate cost lever on the Claude platform after prompt caching. The tax is a real architecture change to async, and you should not pretend otherwise. But for teams running any kind of bulk classification, generation, or analysis, it is straightforwardly half the bill for the same answers.

For more on optimizing Claude in production, see our writeups on prompt caching and tool use patterns.

FAQ#

How much does the Claude Batch API actually cost compared to the synchronous API?#

The Batch API charges 50% of the synchronous API's per-token rate for input, output and special tokens, and the prompt caching discount stacks on top. For 100k classification requests with a 4k-token shared instruction on Claude Haiku 4.5, that is about $425 synchronous versus about $212 in batch, and roughly $36 to $159 in batch with caching depending on the cache hit rate.

What is the maximum batch size I can submit?#

Each batch can contain up to 100,000 requests or 256MB total file size, whichever limit you hit first. For larger jobs, you'll need to shard across multiple batches - a simple approach is to hash your custom_id mod N to distribute rows evenly.

How long does a batch take to complete?#

Anthropic says most batches complete within 1 hour, and requests that are not processed within 24 hours expire. Don't promise users anything faster than 24 hours, but also don't design dashboards that show "still processing" for hours before anyone should actually worry.

Can I use tools and function calling with the Batch API?#

Yes. Your own client tools cannot loop inside a batch: if a row returns a tool call, you run it and submit a follow-up request. Anthropic's server-side tools, such as code execution and MCP connectors, run their loop inside the batch, and a pause_turn result means you continue the turn in a follow-up request.

Do results come back in the same order as my input requests?#

No. Results return in whatever order they finished processing, not the order you submitted them. You must match results to inputs using the custom_id field. Anyone who tries to zip inputs to outputs by position will get incorrect data.

Does prompt caching work with the Batch API?#

Yes, and the caching and batch discounts stack. Because batch requests run concurrently and in any order, cache hits are best-effort; Anthropic cites typical hit rates of 30% to 98%. Put identical cache_control blocks in every request and share as much of the prefix as you can.

How do I handle failed requests within a batch?#

Individual rows can fail without taking down the entire batch. You must iterate through the results file and check each result's type (succeeded, errored, canceled or expired). Build three buckets: succeeded, failed, and missing. Errored rows are eligible for retry on the live API or a follow-up batch. Missing rows almost always indicate a bug in batch construction - duplicate custom_ids, oversized rows, or encoding problems.

Will my batch requests affect my synchronous API rate limits?#

No. Per Anthropic's batch FAQ, the Batches API has its own limits on HTTP requests and on requests waiting to be processed, and batch usage does not affect your Messages API rate limits. Your synchronous traffic still has its own limits and can still see overload errors; see our retry rules for 429, 529 and 503 errors for which responses to back off on and which to stop on.

Continue Reading#

• How to Cut Your LLM API Bill in 2026 - batch discounts across 15 providers, and where batch stacks with prompt caching
• Extended Thinking in Claude: When Deep Reasoning Pays For Itself
• The Harness Is the New Cost Lever: Databricks' Benchmark and Pi's Context Discipline
• Claude Haiku 5.5 Developer Guide - small-model pricing that stacks with the Batch discount
• Claude Fable 5 API: Production Integration Patterns, Rate Limits, and Migration Gotchas - rate limits and migration gotchas for the synchronous side
• Claude - the model family, pricing, and limits on one page
• Where to Access AI Models - official APIs, cloud platforms, and routers compared