Earning the Expansion
A pilot that cannot say what success means was never actually being tested.
Eighty-eight percent of enterprise agent pilots never reach production. That is the most quoted number in enterprise AI this year, and it deserves the same scrutiny this series asks of everything else. Coverage attributes it to Forrester and Anaconda research, and credits the replication to a different set of firms depending on who is writing. This piece could not trace the figure to a public primary document. Forrester’s own published summary of its June report, The State Of Agentic AI, 2026, makes the point without it: roughly three in four enterprise leaders report adopting agentic AI, and few have it running in meaningful production. So 88 percent is used here as what it is, a widely circulated estimate and a convenient shorthand. The eighty-eight and the twelve appear below as names for the pilots that stall and the pilots that ship. The argument rests on evidence a reader can check.
What can be checked points the same direction. Gartner forecast in mid-2025 that more than 40 percent of agentic AI projects will be canceled by the end of 2027, citing escalating costs, unclear business value, and inadequate risk controls. Databricks, working from platform telemetry across more than 20,000 organizations instead of a survey, found that only 19 percent had deployed agents at all, that organizations using evaluation tools put nearly six times as many AI projects into production, and that organizations using AI governance put more than twelve times as many. LangChain’s survey of 1,340 practitioners found that 89 percent had implemented observability for their agents and only 52 percent ran offline evaluations. Most teams can watch their agents. Far fewer can say whether the agents are right.
A reasonable reader will notice that other numbers look far healthier. The same LangChain survey reports 57 percent of respondents with agents in production, and Mayfield’s CXO survey reports 42 percent. Those figures and the grim ones measure different things. One asks whether an organization has any agent running anywhere. The other asks what happens to a given pilot. An enterprise with one triage agent in production and eleven stalled pilots counts as a success in the first kind of survey and as a failure rate above ninety percent in the second. Both are true at once, and the second is the one that decides whether the investment pays.
What Actually Breaks
Three failure modes recur across this year’s reporting on stalled pilots, and each one is a governance failure wearing a technical costume. The breakdown most often quoted, attributed to Forrester, assigns roughly 41 percent of failures to unclear success criteria, 33 percent to insufficient tool or data access, and 26 percent to drift in evaluation coverage. Coverage disagrees on whether those shares describe pilots that never shipped or deployments that shipped and lost money, so the percentages are best read as rough proportions. The categories are the useful part.
Unclear success criteria means the pilot never had a working definition of what production-ready would look like. Teams build the agent, watch it perform reasonably well in a curated demo, and only discover months later, usually when someone in leadership asks whether it is ready to ship, that nobody agreed in advance what ready actually meant. Accuracy thresholds, latency limits, escalation triggers: these are decisions that have to be made before a pilot starts, not questions to answer retroactively once stakeholders are already arguing about whether the results look good enough. A team that cannot answer, on request, what percentage of tasks the agent needs to handle correctly before it ships has not built a pilot with unclear success criteria. It has built a demo with no criteria at all, dressed up as a pilot because a demo alone would not have secured the budget.
Insufficient tool or data access means the pilot succeeded inside an environment production cannot replicate. A pilot given curated, pre-cleaned data and a narrow, cooperative set of test cases will perform beautifully, because the environment was built to let it perform beautifully. Production data is messier, production edge cases are stranger, and a pilot that was never tested against that mess has no evidence it can survive contact with it. This failure mode is invisible until the moment it becomes catastrophic, because a successful pilot demo looks identical whether the underlying capability is real or an artifact of a friendly test environment. A customer-service pilot trained and tested against a clean ticket queue with predictable categories will meet a production queue full of ambiguous, multi-issue, badly worded real tickets, and the gap between those two environments is exactly where a pilot that looked ready stops being ready without anyone noticing until the damage is done.
Evaluation drift means the eval methodology that certified the pilot as working did not survive contact with the actual production query distribution. An eval built around the queries a team anticipated during development frequently bears little resemblance to the queries a system actually receives once real users, with real and unpredictable needs, start using it. A pilot evaluated against the wrong distribution can pass every internal test and still fail its first month in front of real customers, because the test was measuring the wrong thing from the start.
The Profile of the Twelve Percent
The pilots that do reach production, and stay there, share an unusually consistent operating profile, and it is buildable rather than mysterious. It starts with the smallest valuable agent: one task, tightly scoped, three to five tools at most, a hard stopping condition, and a human-in-the-loop checkpoint on any action with real consequence. This is not caution for its own sake; it is the only way to get a clean enough signal, early enough, to know whether the underlying approach actually works before scaling it into a mess that obscures the signal entirely.
The best public evidence on this profile is Measuring Agents in Production, a study of 306 practitioners and 20 deployed systems. It names reliability as the top development challenge, which is worth saying plainly: model reliability is a real constraint. The more useful finding is how the teams already in production deal with it. Sixty-eight percent of their agents execute at most ten steps before a human intervenes, and nearly half execute fewer than five. Seventy percent run on off-the-shelf models with no weight tuning. The study’s own conclusion is that practitioners achieve reliability through system design, not through model advances. The teams that shipped did not wait for a better model. They scoped around the one they had.
Tool constraints follow directly from that scoping. Read-only access before write access, so a misfiring agent’s blast radius is bounded while the team is still learning how it fails. Observable failure modes and explicit error returns, so a failure produces a visible signal rather than a silent, confusing gap in the output. No implicit state, so the agent’s behavior in one interaction does not depend on invisible context accumulated from a previous one, which is one of the hardest categories of bug to diagnose in any agentic system and one of the easiest to avoid through disciplined design from the start.
Defined expansion criteria are the piece most pilots skip entirely, and skipping it is usually what turns a promising narrow pilot into an abandoned one. Scope should expand only when the existing scope has passed a defined evaluation bar, over an extended period, in actual production, with failure modes that are documented rather than merely absent from a small sample. A pilot that performed well for two weeks has not earned expansion. A pilot that has held its evaluation bar for a defined, meaningful stretch of real production traffic has earned it, and the difference between those two thresholds is exactly the difference between the organizations in the 12 percent and the organizations that keep launching pilots that fail unnoticed.
Mature evaluation and observability infrastructure underwrites all three of the above, because none of the other three disciplines mean anything without a reliable way to measure whether they are being followed. This is the same infrastructure the rest of this series keeps returning to, an eval methodology built and instrumented before a system scales, not audited after the fact. The Databricks multiples are the clearest measure of what it is worth: six times the production throughput with evaluation tooling, twelve with governance. An enterprise that has already built that infrastructure for the reasons the earlier pieces in this series describe gets the pilot-to-production discipline largely for free, because the underlying capability transfers no matter which specific problem an organization is solving.
Two Objections Worth Answering Directly
The scope objection. The framework above will strike some readers as too conservative to produce anything commercially interesting, a permanent diet of narrow, low-ambition agents too small to matter at enterprise scale, and a competitive liability against a rival running broader, faster, less cautious pilots. That reading mistakes a starting discipline for a ceiling, and it rests on two assumptions that do not survive contact with how the work actually unfolds. The first assumption is that narrow scope means slow progress. It is the opposite. An enterprise running ten narrowly scoped agents in parallel, each earning its own expansion on its own schedule, is not slower than a competitor running one broad pilot; it is running ten independent experiments on the same timeline the broad pilot spends running one. Smaller batch size is precisely the discipline that moved software delivery from waterfall to continuous release for exactly this reason: smaller units produce faster, more frequent, more independently interpretable feedback, and more total validated capability accumulates per unit of time than one large bet produces, even though any single narrow agent looks modest in isolation.
The second assumption is that broader scope and narrower scope carry comparable risk, differing only in ambition. They do not, and the difference is structural rather than a matter of degree. A broad-scope pilot with wide tool access has no contained failure domain, so when it fails, and by every available account most agent pilots do, the entire initiative fails at once, taking the budget and organizational trust riding on it down with it. A narrow pilot inside a parallel portfolio can fail without threatening the other nine; the organization loses one data point, not the whole initiative. That is not a smaller blast radius in degree; it is the structural difference between a portfolio built to survive individual failures and a single bet that cannot. Defined expansion criteria exist to compound this advantage further: scope is meant to grow, deliberately, once an agent has proven itself against a real evaluation bar under real production conditions. The framework is not an argument for staying small; it is an argument for earning bigger, in parallel, at a velocity a single broad, uncontained pilot structurally cannot match. The pilots that eventually run at genuine enterprise scale, handling meaningful volume across complex workflows, are not the pilots that started broad. They are the pilots that started narrow, proved the discipline held, and expanded on schedule with the evaluation infrastructure already in place to catch problems the broader scope would otherwise have hidden.
Get Nathan Kling’s stories in your inbox
Join Medium for free to get updates from this writer.
The cost objection. The second objection is more honest and harder to wave away: mature evaluation and observability infrastructure is real investment, and most enterprises have not built it. That is true, and it is also the highest-leverage AI investment available to most organizations right now, for a reason that has nothing to do with any single pilot. Eval infrastructure built for one agent transfers to the next one, and the one after that, which means the investment amortizes across every future pilot an organization runs rather than resetting to zero each time. An enterprise that treats this infrastructure as a one-time cost attached to a single project will underinvest in it every time, because no single pilot’s budget can justify the full cost on its own. An enterprise that treats it as standing infrastructure, built once and reused indefinitely, makes an investment that gets cheaper, in effect, with every additional pilot it enables.
Specification, Not Intelligence
Specification, not capability, is the throughline across all three failure modes. A pilot with unclear success criteria was never given a real specification of what it was supposed to do. A pilot with insufficient tool or data access was specified for one environment and deployed into a different one without anyone noticing the gap. A pilot with evaluation drift had a specification that stopped matching reality the moment reality started sending it queries nobody anticipated. In each of the three, the model did what it was actually asked to do. The organization never wrote down, with enough precision, what it meant to ask.
This reframes the actual purpose of the “smallest valuable agent” principle. Narrow scope is not a hedge against a weak model; it is the mechanism by which an organization can write a specification precise enough for a real evaluation bar to measure. A broad, ambitious agent scope makes a precise specification nearly impossible to write, because there are too many dimensions of success to define in advance, which means the pilot’s eventual evaluation will inevitably be vague, and a vague evaluation cannot distinguish a pilot that is actually working from one that merely looks like it is. Start narrow, and the specification is tractable. Prove the specification holds under real production conditions, and the expansion that follows inherits a discipline the broad, ambitious version of the same pilot never had the chance to build.
Building Toward the Twelve Percent, Deliberately
None of this requires new model capability. The organizations on both sides of the production line are buying from the same short list of model providers, and the teams already in production are mostly running those models off the shelf. What separated a pilot that shipped from one that stalled was not the intelligence of the system. It was whether the organization running the pilot built a specification precise enough to test, constrained the agent’s tools enough to bound its failures, and defined, in advance, the evaluation bar that would earn the agent more scope rather than deciding that question after the fact under pressure to show results.
Picture two organizations piloting functionally identical agents on the same underlying model, six months apart. The first defines success loosely, grants broad tool access because narrow access felt like it would slow the demo down, and skips writing an expansion criterion because the team is confident the pilot will obviously earn more scope once stakeholders see it work. The second writes a precise specification before a line of the pilot ships, constrains tool access to exactly what the first task requires, and sets a defined bar the pilot has to clear, over a defined stretch of real production traffic, before its scope grows at all. Eighteen months later, one of these organizations is inside the 12 percent, expanding a proven agent into adjacent workflows with a documented track record behind each expansion. The other is explaining, in a strategic review, why an initiative that looked so promising in the demo never actually shipped. The model was never the variable. The discipline was.
The enterprises succeeding with agents in 2027 will not be the ones with access to better models. Model access is converging fast enough across the industry that it will not remain a durable differentiator for long, if it ever really was one. The enterprises succeeding will be the ones that treated pilot-to-production as a scoping and evaluation discipline from the first pilot onward, earning each expansion of scope the way the twelve percent already do, rather than treating agent deployment as an intelligence problem that a smarter model would eventually solve on its own. Most of the pilots that stall are not failing because their models are not good enough. They are failing because nobody wrote a specification precise enough to prove it.
References
Forrester. “The State Of Agentic AI, 2026.” June 3, 2026. Public summary: forrester.com/blogs/the-state-of-agentic-ai-in-2026-companies-are-chasing-few-are-catching
Gartner. “Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027.” June 25, 2025. gartner.com/en/newsroom/press-releases/2025–06–25
Databricks. “State of AI Agents.” January 2026. databricks.com/resources/ebook/state-of-ai-agents
LangChain. “State of Agent Engineering.” Survey fielded November to December 2025. langchain.com/state-of-agent-engineering
Mayfield. “The Agentic Enterprise in 2026.” January 2026. mayfield.com/the-agentic-enterprise-in-2026
Pan, Melissa Z., et al. “Measuring Agents in Production.” arXiv:2512.04123, December 2025. arxiv.org/abs/2512.04123
Frequently Asked Questions
What does “the eighty-eight percent” actually refer to, and where does the number come from? It is the most widely quoted estimate of the share of enterprise agent pilots that never reach production, attributed in coverage to Forrester and Anaconda research. This piece could not trace it to a public primary document and uses it as shorthand. The argument rests on sources that can be checked: Forrester’s published summary, Gartner’s cancellation forecast, Databricks telemetry, and the LangChain and Measuring Agents in Production studies.
How should an enterprise leader act on this before launching or evaluating a new agent pilot? Start with the smallest valuable agent, one task, three to five tools, a hard stopping condition, and a human checkpoint on any consequential action, and define the evaluation bar a pilot has to clear before its scope expands, in writing, before the pilot begins rather than after results are already in hand.
Other surveys say most companies already have agents in production. Which is right? Both. Surveys that ask whether an organization has any agent in production return figures between 40 and 60 percent. Evidence about what happens to individual pilots is much worse. An organization can have one agent live and a dozen pilots stalled. The first number describes adoption. The second describes whether the pilot portfolio is paying.
Does this scoping discipline just produce agents too narrow to ever be valuable at scale? No. The framework’s defined expansion criteria exist specifically to grow scope once an agent has proven itself against a real evaluation bar under real production conditions, which means narrow scope is a deliberate starting discipline rather than a permanent ceiling, and the pilots that expand successfully are the ones that earned the expansion rather than assumed it.
• • •
This is Article 5 of 6 in the series “The Governance Debt.” Previous: Doing the Math.
