Kill Your Agent Runs Early

Last updated: August 5, 2026

Sixty-six point six, sixty-six point eight, seventy-one point eight. Same model, same benchmark, three ways of running it.

The first number is a coding agent on SWE-bench Verified left to its own devices: 66.6 percent resolution. The second is the same agent killed mid-run by a cheap failure detector and restarted cold: 66.8 percent. The third is the same agent killed and restarted with the interrupted work offered back as an optional overlay: 71.8 percent. Five points of resolution for the price of knowing when to stop and carrying a diff across the restart (FailFast-RestartSmart, arXiv:2608.03222).

The detector in question is a 0.6 billion parameter model. Not the frontier, not a router, not a judge: a model smaller than most embeddings pipelines, trained to predict failure from observable action prefixes alone, no logits, no hidden states. It transfers to policies it never trained on, including closed-API models. One cheap monitor gating a fleet.

Here is what we think is happening, and we want to defend it properly: the next lever in agent quality is not the model and not the context window. It is the run lifecycle. When to kill a run, what to carry across the kill, and what to gate deterministically are now the measured wins, and the first production-scale trace of the agentic workload explains why in one number.

What we argued before, and what changed#

We have been building a position on this site across the last week. On August 1 we argued the benchmark numbers are lying to you, with double-digit noise at every layer of the eval stack (your-benchmark-is-lying-to-you). Later that day we argued the fix is architectural, not model-side: verification has to move into the artifact, not sit on top of it (the-benchmark-fix-is-architectural). On August 3 we argued the judge is leaving the loop, because evidence gates and verifiable structures beat LLM verdicts at every point a loop needs a decision (the-judge-leaves-the-loop).

The evidence this week moves the argument one level down. The judge posts were about how agents are graded and gated. The new batch is about how runs are born, kept alive, and killed - and it lands with the first production-scale measurement of the workload itself. GitHub sampled Copilot traces from June 2026: 3.2 million users, 13 million sessions, 761 million LLM calls, 95 trillion tokens (arXiv:2608.00101). For the first time we know what agentic coding looks like at planetary scale, and the shape is nothing like the chatbot workloads serving infrastructure was built for.

Agentic sessions are sparse user turns. Each turn unfolds into an autonomous loop of LLM calls, almost always coupled with tool execution. The user goes quiet; the agent works. And here is the number that should change how you think about every agent run you have ever watched grind: KV-cache hit rate averages 90 percent inside a turn, and 55 percent across turn boundaries. Context compaction and model switches destroy most of the cache. The context you are protecting when you let a run continue is the thing the serving layer has already half-thrown away.

The marginal unit of agent compute is the expensive one#

This is not the first time the numbers pointed this way. At equal token cost, every one of 18 self-inspection comparisons loses to plain repeated sampling - reflection is a worse use of the second unit of compute than sampling again (arXiv:2607.28576). Repair-only policies build much more accurate simulators that play worse (83.07 versus 88.49 on the ARC-AGI-3 measure) - more repair compute is not just wasted, it is negatively correlated with decision quality (Tycho, arXiv:2607.28287). Fixed reasoning effort loses to per-call escalation-on-stall at equal cost in optimization loops (ARES, arXiv:2607.27879). We covered the shape of this in the judge post: the marginal unit of agent compute saturates, and on a growing list of axes it goes negative.

What the new batch adds is the production-grade execution of the same idea, and the repair primitives that make it safe.

The kill. FailFast's prefix monitor, trained on terminal plus dense fail-to-pass supervision, saves 14.6 to 20.4 percent of execution tokens at a 5 percent false-positive target, and beats the per-step AgentStop adaptation on the same model (20.4 versus 12.5 percent). At a looser 25 percent false-positive target, restart-with-overlay is where the resolution gain shows up: 71.8 versus 66.8 percent for cold restart. Early termination is not lossy when the partial state is offered back. It is lossy exactly when you throw the state away.

The repair loop. A separate group ran a detect-and-repair loop over 2,823 committed agent episodes across three frameworks and four models (arXiv:2608.02464). The detector is a one-class echo-state-network ensemble with CUSUM alarms, trained only on healthy runs, about 200 microseconds per step - three orders of magnitude cheaper than an LLM judge call - and it catches 0.71 of failures at a 5 percent false-alarm budget. On top sits a deterministic verification layer that recomputes a run's stated total from the tool results it actually received: it catches 60 percent of failures at 0 of 63 false positives, with zero calibration, and it transfers across models unchanged. Then the repair: roll back the flagged episode and re-run it, which recovers 45 percent of failures against a 16 percent resampling control (p = 0.0005), lifting task success from 52 to 73 percent for about one extra model call per run.

The state. The most surprising result of the batch is Ledger (arXiv:2608.00808): a deterministic runtime layer, zero LLM calls, that distills a long-horizon coding agent's history into an explicit execution state - what it has observed, modified, and attempted - and applies it at two boundaries. An inform path appends a compact state view before each model step; a govern path checks proposed commands against the ledger and returns still-valid earlier results instead of re-executing. Across all 500 SWE-bench Verified instances it raises Pass@1 from 56.2 to 64.2 percent (GPT-5 mini) and 75.8 to 81.0 percent (MiniMax M2.5) while cutting cost by about 30 percent. Attached to OpenAI Codex it adds 3.4 points at 24.4 percent lower cost. The ablations attribute most of the resolution gain to govern and most of the efficiency gain to inform: the run's own history, rendered machine-checkable, is both a correctness fix and a cost cut.

And the failure class it kills. SWE-Touch stress-tests coding agents in shared workspaces by injecting Counter-Edits - plausible user edits that conflict with the task - when agents reach the relevant code (arXiv:2608.02499). Across nine models the Counter-Edits lower resolve rates by 7.7 points, and the trajectory analysis pins the cause: weak workspace-state awareness. Agents keep conflicting code or replace it without re-inspecting the repository and validating with targeted tests. Ledger's govern path is the systems answer to exactly that gap, and the two papers landed the same day: the failure class and the fix measured independently.

The same shape shows up when a run hands work off. On repository-level code QA, semantic search against a prebuilt index answered 65.2 percent of questions versus 46.2 percent for deep agentic search with a grep subagent in an isolated context - the pattern adopted by Claude Code, Codex, and Antigravity - at less than half the cost per correct answer, and 41.8 percent of the agentic failures happened at the planner-to-subagent handoff, usually silently, ending in a fluent confident wrong answer (arXiv:2608.01507). The most expensive lifecycle decision is not how many steps a run takes. It is where the run's working memory lives when it hands off.

The mechanism: state, not tokens, is what should survive#

Here is the part nobody has quite said out loud, and it is the reason we think this is a trend and not a week of lucky benchmarks. Put the trace together with the repair loop and the state layer, and a consistent picture falls out.

Continuing a run is an investment in its context. The Copilot trace prices that context: inside a turn it is worth keeping, with 90 percent cache hits; across a turn boundary it is worth 55 percent, and model switches and compaction destroy it further. The run you refuse to kill is the run whose working memory the serving layer is already discarding. So "should I let this run continue" is secretly a question about what survives, and the answer the new papers converge on is that the thing which survives should not be tokens at all. It should be a compact, machine-checkable state: what was observed, what was modified, what was attempted, what was verified. Ledger shows that state layer costs nothing to build (zero LLM calls), buys 8 points of Pass@1, and cuts the bill by a third. FailFast shows the overlay can ride across a restart for free. The telemetry paper shows the whole detect-verify-repair loop can run with no LLM judge in the stack at all.

The durable unit of an agent run is shifting from the token stream to the execution-state ledger. That is why restart-with-state beats both continuing and cold restart: continuing protects a dying cache, cold restart throws away the one thing worth keeping. Kill the run, keep the state, spend the saved tokens on a fresh trajectory. Checkpoint as state, not as cache.

We have logged this as a combination thesis internally, and it is the load-bearing claim of this piece, so let us be precise about what it predicts and what would falsify it.

The bet#

By end of 2027, we expect restart-with-state-overlay to be a named repair primitive in mainstream agent harnesses, cheap prefix monitors on the hot path to be a documented pattern in fleet tuning guides, and "how much compute" to stop being the headline lever in agent-cost content - replaced by when-to-stop, what-to-carry, and what-to-gate. The economics run the right way: the monitor is a 0.6B model, the state layer is deterministic, the gate is free, and every one of them pays for itself in tokens. We are wrong if the platforms keep shipping longer-horizon modes and bigger context as the headline quality lever and kill/restart stays a research artifact. That is a graded call, and we will grade it.

The counter-case, honestly#

Every leg of this is a single result from a single research week, and the honest counter-case is stronger than we would like.

First, the curve is per-task. SKIMIX found harness-time gains are front-loaded and task-dependent - negative on multiple-choice, real on open-ended mathematical reasoning - and agent-count scaling is non-monotonic (arXiv:2607.27994). Some task classes genuinely reward more compute, and "kill early" guidance that ignores task class will be wrong on exactly the tasks where patience pays.

Second, the monitor has a deployment tax. The telemetry paper is admirably honest: its echo-state detector needs per-deployment healthy-run calibration, and cold it sits at AUROC 0.527 versus 0.885 recalibrated. A fleet that skips calibration is running on a coin flip. For small teams the calibration burden may exceed the token savings.

Third, the kill decision is not uniformly cheap. The 90 percent intra-turn cache hit rate means the early part of a turn is exactly where continuing is still cheap. Kill too early and you forfeit banked work; the monitor's false-positive budget is the price of that mistake. The upside case runs at 25 percent false positives, which is a lot of killed runs. The trade is a dial, not a theorem.

Fourth, the long-horizon measurement problem is unsolved. A position paper this week showed headline long-horizon gaps currently mix ordinary error compounding, genuinely harder decisions, and context rot, and cannot be decomposed without stage-baseline controls (arXiv:2607.27283). Our kill-early reading inherits that ambiguity: some of what looks like "continuing is wasteful" may be "this particular task is beyond the model," which no amount of lifecycle engineering fixes.

Fifth, the frontier could reset the curve. A model with reliable self-verdicts - the SVR-style trained stopping signal we covered in the judge post - makes external monitors redundant, and a model that plans context use internally could make the 55 percent cross-turn cache number obsolete. The cheap monitor predicts the current policy generation. Nothing here survives contact with a model that knows when it is wrong.

What developers should do#

1. Instrument runs before you lengthen them. A prefix monitor or a deterministic state check is cheaper than one extra reasoning step, and it is the thing every result above starts from. If you cannot tell a doomed run from a slow one, you are not running agents, you are gambling.

2. Make restart-with-state your default repair. When a run fails, do not feed its corpse more context. Kill it, keep the diff and the ledger, start fresh with the overlay. The FailFast numbers say the overlay is the entire difference between 71.8 and 66.8.

3. Carry state, not tokens, across boundaries. Ledger's inform/govern pattern is deterministic and free: observed, modified, attempted. If your agent re-executes work it already did, that is a harness bug with a measured fix, not a model limitation.

4. Trust deterministic verification over monitors where both exist. The telemetry paper's deterministic layer catches 60 percent of failures at zero false positives with no calibration; the monitor needs calibration and alarms. Use the free layer first, always. This is the same rule the judge post ended on.

5. Budget for the false-positive dial and say which side you are on: kill-early economics live or die on the monitor's false-alarm rate, and the papers above publish it. Do the same.

None of this means the model stops mattering. It means the model's job is narrowing to the parts nobody has found lifecycle for yet, and the people who get good at run governance - when to kill, what to carry, what to gate - are the people who will run fleets in 2027. This is one thread of our developing long-range scenario: the harness layer keeps absorbing the quality lever, and the agent run becomes a designed artifact with a birth, a kill condition, and a state that outlives it.

FAQ#

Does killing and restarting a coding agent run improve results?#

Only when the interrupted work is carried across the restart. In FailFast-RestartSmart, the same agent on SWE-bench Verified scored 66.6 percent left alone, 66.8 percent when killed by a failure detector and restarted cold, and 71.8 percent when restarted with the interrupted work offered back as an optional overlay.

How small can an agent failure detector be?#

FailFast's detector is a 0.6 billion parameter model that predicts failure from observable action prefixes alone, with no logits or hidden states, and transfers to policies it never trained on, including closed-API models. It saves 14.6 to 20.4 percent of execution tokens at a 5 percent false-positive target.

What is Ledger and how does it help coding agents?#

Ledger is a deterministic runtime layer, with zero LLM calls, that distills a coding agent's history into an explicit execution state: what it has observed, modified, and attempted. Across all 500 SWE-bench Verified instances it raised Pass@1 from 56.2 to 64.2 percent (GPT-5 mini) and 75.8 to 81.0 percent (MiniMax M2.5) while cutting cost by about 30 percent.

Why does context matter less across agent turn boundaries?#

GitHub's Copilot trace of 761 million LLM calls found KV-cache hit rate averages 90 percent inside a turn but 55 percent across turn boundaries, because context compaction and model switches destroy most of the cache. The context a long run protects is already half-discarded by the serving layer.

When is killing an agent run early the wrong call?#

Gains are task-dependent: SKIMIX found harness-time gains negative on multiple-choice but real on open-ended mathematical reasoning. Monitors also need calibration, with the echo-state detector at AUROC 0.527 cold versus 0.885 recalibrated, and killing too early forfeits work banked inside a turn.

Continue Reading#

• The Text Is a Control Surface - the same placement rule one level down: re-ground after the lossy stage, preserve relationships, and price the text the model reads
• The Judge Is Leaving the Agent Loop - related deep dive
• The Fix for Broken Benchmarks Is Architecture, Not Smarter Models - related deep dive
• Your Benchmark Is Lying to You - related deep dive
• Agent Swarms Need Receipts - related deep dive
• DeepSeek V4 Economics: Cost, Quality, and the Frontier of Agentic Coding - DeepSeek V4 Economics: The Cost-Quality Frontier for Agentic Coding in
• The Quiet Tax on Your Cheap Agent Tier - compression's invisible tax, and why serving-side levers come first
• The Response Looked Right. The Work Was Not Done. - the completion side of the lifecycle: agents claim done they have not earned, and evidence-carrying termination now certifies the stop

Sources#

• FailFast-RestartSmart: prefix-monitor failure prediction and restart-with-overlay repair - arXiv
• First production-scale characterization of the agentic coding workload from Copilot traces - arXiv
• Agent failure detection from step telemetry with rollback-repair - arXiv
• Ledger: explicit execution state as a runtime layer - arXiv
• SWE-Touch: user edits mid-task and workspace-state awareness - arXiv
• Semantic search beats subagent-grep search on repo QA - arXiv
• Sample more, reflect less: self-inspection loses to repeated sampling - arXiv
• Tycho: simulator accuracy is not decision quality - arXiv
• ARES: adaptive effort escalation in optimization loops - arXiv
• SKIMIX: skill-mixture collaboration and task-dependent gains - arXiv
• Horizon residual: separating long-horizon degradation from error compounding - arXiv