How Much Does It Cost to Run an LLM in Production? A Real Breakdown

Token economics, a worked chatbot bill, the cost drivers nobody warns you about, and the levers that actually cut the invoice.

The Question That Kills Most AI Projects

I’ve sat in more than one meeting in Singapore where a brilliant LLM demo got a standing ovation on Friday and got quietly killed the following Wednesday. Not because it didn’t work. It worked beautifully. It got killed because someone in finance finally asked the question nobody on the engineering side had modeled: what does this cost at scale?

For seventeen years as a Java architect, I could reason about cost with a spreadsheet. CPU cores, RAM, a database license, some bandwidth. Costs were lumpy but predictable — you provisioned a box, and the box cost the same whether it served ten requests or ten thousand. LLMs broke that intuition completely. With an LLM API, every single request costs money, and the cost is proportional to how much text goes in and how much comes out. There is no free idle capacity. The meter runs on every token.

That’s the mental shift. An LLM in production isn’t a server you rent. It’s a taxi with the meter always running. And if you don’t understand…