Deterministic AI at Work: The LLM Is the Most Unreliable Function in My Pipeline, So I Treat It Like One
Stop trying to make the model deterministic. You can’t. Wrap it in code that is, and let the AI touch only the part that’s allowed to be fuzzy.
The first time I let an AI tool draft a field mapping for a payment flow, it gave me something that looked perfect. Then I ran the same prompt the next morning and got a different answer. Different field names. A different assumption about whether the pacs.008 carried remittance data inline or by reference. Both sounded equally authoritative. Only one matched the message we were actually building against.
A lot of people respond to that problem by getting better at prompting. Pin the context, ask for assumptions first, give it a template. That helps, and I do all of it. But it is the wrong layer to solve the problem at, and as someone who codes, it always bothered me that the best I could claim was “more consistent.” More consistent is not deterministic. If I regenerate and the wording changes, it was never deterministic. It was just lower variance.
So I stopped trying to make the model behave. A large language model is a probability engine. Asking it to be deterministic is like asking a random number generator to be predictable if you phrase the request politely. The fix is not better phrasing. The fix is to stop putting the model in charge of anything that needs to be exact.
The way I build now, the LLM is one untrusted function call sitting inside a pipeline of deterministic code. The code does everything that must be correct. The model only fills the narrow gap that is genuinely fuzzy, and its output gets validated by code before it counts. The system is deterministic even though the model never is. That distinction is the whole article.
If you want the prompt patterns I use inside these pipelines, they are in The Technical BA Prompt Toolkit, but the architecture matters more than the prompts, so let me show you the architecture.
The mental model: the LLM is an unreliable function
Picture any function you would write. It takes typed inputs, it returns a typed output, and you can reason about it. Now imagine a function that ignores its type signature roughly ten percent of the time, occasionally invents a return value that looks correct but is not, and gives you slightly different output on identical input. You would never let that function write directly to production. You would wrap it. You would validate everything it returned, reject anything malformed, and constrain its inputs so tightly that the space of plausible wrong answers shrank to almost nothing.
That is exactly what an LLM is, and exactly how I treat it. It is the unreliable function. My job is to build the reliable code around it so that the unreliability has nowhere to escape to.
This reframing does most of the work. The moment you stop seeing the chat window as the product and start seeing the model as a single call inside a larger deterministic process, every decision gets clearer. What goes before the call. What the call is actually allowed to do. What checks the output before it is trusted. Three layers, and the model only owns the middle one.
Layer one: deterministic code prepares the input
The model should never have to know anything. It should be told everything, and the telling should be done by code, not by you typing from memory.
For a payment field mapping, I do not describe the message to the model. I write a script that reads the actual ISO 20022 message definition and extracts the field names, types, cardinality, and required flags into a clean structured list. That extraction is pure code. It produces the same list every time because it is parsing a file, not guessing. The model never sees the raw spec and never gets to interpret it. It sees a fixed, code-generated list of facts.
This is the part where being technical is the entire advantage. I can read the schema, query the database, and pull the real Kafka topic configuration into the prompt programmatically. Someone who only writes prose has to describe the schema in English and hope the model reconstructs it correctly, which reintroduces exactly the non-determinism we are trying to kill. If you analyze, code, test, and support, you can build the input layer in actual code. I broke down the specific skills that make this possible in The Technical Skills Guide for BAs, because parsing a schema and feeding it to a model is a different muscle than writing a good sentence.
The rule for this layer is simple. Anything that can be computed must be computed, not prompted. Every fact you hand the model as deterministic code is a fact it can no longer get wrong.
Layer two: the LLM does only the fuzzy part
Now the model gets a job, and the job is deliberately small. It does the one thing that is genuinely hard to do in code: turning structured facts into well-written human language, or making a judgment call that has no single right answer.
Take a Kafka consumer validation checklist. The field list, the types, the required flags, all of that came from layer one as deterministic code. I do not ask the model to figure out the fields. I hand it the fields and ask only for the fuzzy bit: given these exact fields, write a plain-English validation description for each one. That is something code is bad at and the model is good at. It is also something where variation does not hurt me. If the wording of a description changes slightly between runs, nobody’s settlement breaks. The facts underneath are fixed by code.
Get Ahmed Squalli’s stories in your inbox
Join Medium for free to get updates from this writer.
The discipline here is to keep shrinking the model’s job until it touches nothing that must be exact. Every time I catch the model doing something that could be deterministic, I move that thing into layer one. The ideal is a model call so narrow that it is almost impossible for it to produce something dangerous, because everything dangerous has already been handled by code on either side of it. I cover this kind of scoping for event-driven systems specifically in Automate Kafka Validation with Postman.
Layer three: deterministic code validates the output
This is the layer people skip, and skipping it is why their pipelines drift. The model’s output is not trusted. It is checked by code before it is allowed to mean anything.
When the model returns those validation descriptions, a script verifies that every description it produced maps to a real field from layer one, and rejects or flags any description that references a field that does not exist. If the model hallucinated a field, the code catches it mechanically, not by my eyeball. The output then gets dropped into a fixed template, also by code, so the structure is identical every run. The model never controls the shape of the deliverable. It only fills slots that the code has already validated.
This is the same pattern as a linter, a schema validator, or a unit test. You do not ask the contributor to be careful. You enforce correctness with code that runs every time, regardless of how confident the contributor sounds. And confidence is exactly the trap with LLMs. The output sounds polished whether or not it is right, so the only safe move is to make verification mechanical and unconditional. Polish and correctness are completely uncorrelated in a language model.
The bonus is accountability. When something ships and someone asks where it came from, “the AI wrote it” is not a defense. “Code extracted the fields from the spec, the model drafted descriptions, and code validated every one back against the schema, here is the trace” is. The deterministic layers are also your audit trail.
A worked example: the same task, two architectures
Let me make this concrete the way I actually validated it. I took one real task, generating a validation checklist for a payment event consumer, and built it both ways.
The chat-only way was a single request: write a validation checklist for our payment event consumer, with the schema pasted into the prompt. I ran it three times. Three meaningfully different checklists. One dropped schema validation entirely. One invented a retry policy we do not use. All plausible, all subtly wrong, and the wrongness was load-bearing.
The pipeline way had three layers. Code parsed the Avro schema and the topic config into a structured field list. The model received that list and was asked only to write a description per field. Code then checked every returned description against the field list and rejected anything off-schema, then rendered the result into a fixed template. I ran it three times. The outputs were effectively identical, and every line traced to something code had produced or validated. The variation collapsed, not because the model improved between runs, but because I had removed every place it was allowed to vary on anything that mattered.
That is the thesis in one experiment. The model is not the part you make reliable. The model is the part you contain.
When the chat window is still fine
I am not claiming everything needs a pipeline. Most of my AI use is still a chat window, and that is correct, because most of it is not producing a deliverable that ships.
When I am brainstorming failure modes, exploring edge cases, or asking what could go wrong with a design, I want the model loose and creative. Running the same “what are the failure modes here” question three times and getting three different sets of ideas is the point, because I am mining for coverage, not generating an artifact. There is nothing to make deterministic there, and trying to would just make the tool worse.
The skill is knowing which mode you are in. Exploratory, throwaway, divergent work belongs in the chat window where variation is a feature. Convergent work that produces the one artifact that ships belongs in a deterministic pipeline where variation is a defect. The mistake I see most is people using the chat window to produce things that should have gone through a pipeline, then shipping the brainstorm.
Loose chat for thinking. Code-wrapped model for shipping.
This is just software engineering applied to a new dependency
None of this is exotic. You took an unreliable third-party dependency, you put a clean deterministic interface around it, you validated everything it returned, and you constrained its inputs so it could not surprise you. We have done this with flaky APIs, with untrusted user input, and with external services for decades. An LLM is just the newest unreliable dependency, and it happens to be unreliable in a more articulate, more confident way than the ones before it.
The people who get burned by AI at work are the ones who let the model be the whole system. The ones who get a real edge are the ones who make it a single function call inside a system they actually control. I have built most of my approach to this into the products at The Tech BA Toolkit, and if you want the templates and patterns in one place, they are in The Complete Tech BA Bundle. But you can start with one rule tomorrow: anything in your AI workflow that can be done by code, do it by code, and let the model touch only what is left.
Ahmed is a Senior Technical Business Analyst with 10+ years in banking and payments. He builds practical guides and tools for analysts at The Tech BA Toolkit.
