Fine-Tuning vs RAG vs Prompting — The Decision Nobody Explains

Three ways to customise an LLM, one giant misconception that wastes budgets, and a decision framework that actually tells you which to reach for.

The Question Everyone Asks Wrong

“We want the model to know our company’s data — should we fine-tune it?”

I’ve heard some version of that question in a dozen meetings, and the answer is almost always “no, and you’re asking the wrong question.” Not because fine-tuning is bad, but because most people reach for it to solve a problem it doesn’t solve, while ignoring the tool that actually does.

There are three ways to make a general-purpose LLM behave the way you need it to: prompting, RAG (Retrieval Augmented Generation), and fine-tuning. They are not three tiers of the same thing where fine-tuning is the “serious” option and prompting is for beginners. They solve fundamentally different problems, cost wildly different amounts, and combine in ways that most explanations gloss over.

I spent 17 years as a Java architect making build-vs-buy and cost-vs-benefit calls, and this decision has the same shape: the expensive option is seductive precisely because it feels like “doing it…