Your Agents Can Read Everything and Understand Nothing
Part 1 of the series What Your Agents Don’t Know.
For this series I built the data warehouse of a fictional freight broker, Tessaway Freight, and put an AI agent to work on it as a data analyst. Three systems feed the warehouse, the way they would at a real broker: transportation management, carrier billing, and the portal customers log into. Fifteen tables, about 11,500 load records, nine months of business.
Then I asked the question every operations chief asks on the first of the month: what was our on-time delivery rate in September?
The agent said 91.52%. It showed its arithmetic (1,004 on-time shipments out of 1,097 delivered), and the arithmetic was right. The answer wasn’t. Tessaway’s on-time rate for September was 87.44%.
I asked that question twenty times, ten on each of two Gemini models. Not one run came back with 87.44. Thirteen gave a confident number, every one of them between 91.5 and 92.1. The other seven stopped and asked me what I meant by on-time.
I set the warehouse up to do exactly this, because I wanted to measure something most agent programs take on faith: if an agent can read every table you have, how much of your business does it understand?
The data was not the problem
When an agent gives a bad answer, the reflex is to blame the data. So I took that excuse away.
Every table in Tessaway’s warehouse is complete. Every key resolves. Every join works, and there are tests in the repo that prove it. It’s the kind of warehouse a data team would look at and call ready.
In my last series, an agent got into trouble because two price lists disagreed and nothing told it which was current. This is a different problem, and a harder one. Nothing in this warehouse disagrees with anything else. Every number in it is true. The agent still couldn’t tell me how the company did last month.
Three systems, three meanings of on-time
Ask three people at a freight broker whether a load was on time and you’ll get three answers, each one correct.
The dispatcher means the truck arrived inside its appointment window. By that rule, September was 84.41%.
The customer portal means the freight was delivered on or before the date the customer asked for. It compares dates, not appointment times, so a truck that shows up four hours past its slot on the right day counts. By that rule, September was 91.52%.
The contract is the one Tessaway reports to customers and gets held to. It uses the dispatcher’s rule with one change: a late load that the shipper delayed (freight wasn’t ready, dock was closed) is left out of the measure. By that rule, September was 87.44%. That’s the official number.
The portal’s table has a column named for exactly this, an on-time indicator with a Y or an N in it. The agent found the column, counted the Y’s, and reported the result. I can’t fault the logic. A sharp new analyst would do the same thing in week one. The difference is that by week two the analyst has been corrected by somebody in operations, and the agent never will be, because nothing it can read says the portal’s flag isn’t the company’s number.
What I asked, and how I scored it
On-time is one of six gaps like this in the warehouse. The others are just as ordinary: what counts as delivered, what goes into margin, who the customer is when one company ships from six sites, what counts as one load, and what happens to Canadian dollars and time zones. Each is written down in the repo, and each was written down before any agent ran.
I wrote 24 questions an executive would ask. Six are plain lookups that need no definition (how many carriers are on file, which city a facility is in). They’re the control group. The other 18 can’t be answered correctly without knowing what Tessaway means by a word.
The agent got what most agents get on day one: table names, column names, column types, and one tool that runs a read-only query. It could run up to eight queries per question. It was also told, in so many words, that it could stop and ask for clarification instead of answering.
Each question ran ten times on Gemini 3.8 Flash and ten times on Gemini 3.1 Pro (in preview as I write this), the stronger of the two. That’s 480 runs. Every reply was scored by plain code against an answer I’d worked out in SQL beforehand, and landed in one of three bins: right, confidently wrong, or declined.
What came back
The six lookups went 120 for 120. So the agent can find its way around the tables and write correct SQL. Whatever goes wrong next, it isn’t that.
On the 18 questions that need a definition, Flash was right 25% of the time. It was confidently wrong 11% of the time. And 64% of the time it declined: it stopped, usually with a tidy list of the things it would need to know.
I expected more wrong answers than that, and I’ll take the result I got over the one I predicted. What it shows is two separate ways to fail.
The stall. Asked for Q3 revenue, Flash declined ten times out of ten. Its questions were good ones: recognize revenue by invoice date or delivery date, linehaul only or with fuel and accessorials, before or after credit memos? Every one of those has an answer at Tessaway. None of the answers is in the warehouse.
Get Robert H Mowery III’s stories in your inbox
Join Medium for free to get updates from this writer.
The guess. Asked how many September deliveries were late, the two models between them answered 93 in fifteen of twenty runs. That’s the portal’s count. The contract’s count is 133.
A wrong answer looks exactly like a right one
This is one reply (minus its final answer line) to the question of how many loads Tessaway booked in September:
“To find the number of loads booked in September 2026, we count the loads created during that month based on the creation timestamp (
crt_ts), making sure to exclude any test or voided records.There were 1,232 valid loads created (booked) in September 2026.”
It reads like care. The agent noticed the test loads and the voided ones and took them out, which is more than some dashboards do. But Tessaway booked 1,170 loads in September. When a carrier accepts a load and then falls off, the load gets tendered again and the system writes a second row, so 62 loads are in that table twice.
The stronger model gave 1,232 on all ten runs. I’d flag that for anyone who checks an agent by asking it twice. Ten identical answers feel like confirmation, and here they’re one mistake made ten times.
Across both models there were 74 confidently wrong answers. I had the code check each one for any sign that the agent knew it was choosing an interpretation: a word like “assume,” “interpret,” or “definition.” One of the 74 had one. It was an on-time answer, and it described the portal’s rule as “the standard definition.” There’s no standard definition anywhere in that warehouse. The agent supplied one.
And when I named the definition in the question (what on-time rate did customers see in the portal?), Flash got it right ten times out of ten. The agent can compute any of these numbers. It has no way to know which one you mean, and nobody in your company says “per the contract definition” when they ask how September went.
The stronger model was wrong more often
The natural fix is a more capable model, so I ran the same 240 runs on Gemini 3.1 Pro.
On the 18 definition questions it was right 41% of the time, up from 25%. That’s the improvement you pay for. It was also confidently wrong 30% of the time, up from 11%.
Compare the two models’ totals. Across all 240 runs, the stronger model declined 63 fewer times than the smaller one. Over the same runs its right answers went up by 29 and its wrong answers went up by 34. More than half of what the upgrade bought was a confident mistake.
I can’t see inside either model, so I’ll describe only what the runs show. The stronger one is better at finding a route through fifteen tables to a number. Where the route forks (portal flag or appointment window, shipper row or parent account), it picks a branch and keeps going. Asked how many customers shipped in Q3, it answered 68 in five of ten runs, counting shipping sites. Tessaway has 36 customers who shipped that quarter. The smaller model declined that question every time.
So the cautious model costs you time, and the capable one costs you trust. An agent that asks a good clarifying question two times out of three is tolerable in a pilot, with a person there to answer. Put it in a workflow that runs at 2 a.m. and the question goes to nobody. And an agent that’s wrong three times in ten, in the same confident voice it uses when it’s right, is one you end up checking by hand every time, which is the job you bought it to do.
Access is not meaning
Every definition that decided these answers exists. Someone at a company like Tessaway knows all of them: the operations lead knows what on-time means in the contract, the controller knows when revenue is recognized, the account managers know that Harlow Dairy is Northgate Foods under another name.
It’s in their heads, in a contract PDF, in the close checklist. The warehouse stores what happened. Nobody stored what it means, because until now every reader of that warehouse was a person who could walk down the hall and ask.
One fictional company and two models aren’t a survey. But the surveys point the same way. In an August 2026 VentureBeat poll, 64% of 130 respondents said they’d traced a confidently wrong agent answer to missing or inconsistent business context in the prior six months. KPMG’s Q3 2026 pulse of 314 U.S. executives put data readiness and access at the top of the list of barriers to deploying agents. I’d add one caution to that last finding. Tessaway’s data was ready by any test a data team would run. Ready and understood turned out to be different things.
What this costs
The cost shows up in three places.
Compute. A lookup cost about half a cent on Flash. A question that needed a definition cost about four cents, because the agent spent all eight of its queries looking for a rule the tables don’t contain. On the stronger model those were about one cent and nine cents. The agent works hardest on exactly the questions it can’t get right.
Supervision. Every stall is a question routed to a person, and every answer you can’t trust is an answer a person re-derives. Either way the human supervision ratio (how many people it takes to keep the agents honest) goes the wrong direction.
Decisions. A 91.52% on-time rate goes into a quarterly review with a customer whose contract says 87.44. A count of 1,232 booked loads goes into a capacity plan that should have said 1,170. What those cost depends on your business, and I won’t invent a number for it.
The query cap shaped some of this. Most of the definition questions used all eight queries, and a small share of the declines (16 of 167) said they’d run out. Some stalls were the cap talking. I kept the cap because production agents have budgets too.
Five questions for your team
- Take your five most-used metrics. Where is each one’s definition written down in a form a machine can read? A slide and a wiki page don’t count.
- Which business terms mean different things in different systems, and who knows that? If the answer is a person’s name, that person is a dependency your agents can’t reach.
- When an agent answers a numbers question, can it say which definition it used? If it can’t, you can’t audit it.
- How often do your agents say “which one do you mean?” Never is a bad sign. So is always.
- Has anyone tested your agents on questions where the right answer is already known? Until someone has, accuracy is an impression.
If most of those come back empty, the gap isn’t in the model and it isn’t in the data. It’s the layer between them, and the rest of this series builds it.
Next in the series: I write the meaning down three ways, as a semantic layer, an ontology and a knowledge graph, and run these same 24 questions again. One of the three did the work.
The code behind every number in this article is public, and my Substack companion walks through it line by line.
Next: Part 2, I Built an Ontology, a Semantic Layer and a Knowledge Graph. One of Them Did the Work.
