The Calculator Must Win the Argument
AI is great with words. It's terrible with math.
The Architecture Series · 8 minute read · By Ed | Founder & CEO, DealiOS
Here's a fun experiment. Ask ChatGPT to calculate the total cost of a 3-year enterprise contract with a 12% annual escalator, a volume discount that kicks in at 15 seats, an onboarding fee waived above the Business tier, and a partner discount of 20% on monthly but not onboarding.
It will give you a number. Run it again. Different number. Third time. Third number. Three attempts, three answers, all delivered with the confidence of a man who brought receipts. Except the receipts are from three different restaurants.
This is not a bug. This is the fundamental architecture of how large language models work. And if your business runs on numbers that matter, it's the most important thing you'll read this week.
Three Answers, Three Restaurants
LLMs don't calculate. They predict what a correct-looking number would be. There's a difference, and it's the difference between an accountant and someone who watched a lot of accountants on television.
The AI doesn't do math. It does math cosplay.
When you ask an LLM to compute a pricing scenario, it's not building a dependency graph. It's not tracing how a seat count change affects volume thresholds, which affects per-seat pricing, which affects the annual total, which affects the escalator base, which affects years two and three. It's predicting the next token. And tokens that look like plausible dollar amounts show up in the output with exactly the same confidence as tokens that represent real calculations.
A 2025 study published in npj Digital Medicine found that GPT-4o's hallucination rate hit 53% under systematic testing conditions. Not on trick questions. Not on obscure trivia. On structured tasks where the model was given source material and asked to work with it faithfully. More than half the time, it invented information instead (npj Digital Medicine, 2025).
The AI doesn't do math. It does math cosplay.
And the scariest part isn't the wrong answer. It's the delivery. MIT researchers found in 2025 that AI models are 34% more likely to use language expressing certainty and confidence when generating incorrect information than when providing accurate responses (MIT, 2025). The more wrong the AI is, the more sure it sounds. It doesn't hedge. It doesn't say "I think this might be $14,400." It says "$14,400" the way a veteran CFO says it after running the numbers twice. Except no numbers were run. A number was predicted based on what numbers near other numbers usually look like.
That's not a rounding error. That's a category error. And in business, category errors have dollar signs attached.
The Dependency Problem
Business math isn't arithmetic. It's a directed acyclic graph with money at every node.
Consider a standard enterprise deal. Seat count determines the pricing tier. The pricing tier determines per-seat rates. But there's a volume discount that activates at a threshold, and that discount applies to the base rate, not the tier rate. There's a partner discount that stacks on monthly licensing but not on implementation. Implementation fees are waived above a certain tier but only for annual contracts. The annual escalator compounds on the post-discount total, not the list price.
Change the seat count by one and potentially every downstream number changes. Not because arithmetic is hard. Because the relationships between numbers are hard. This is a dependency graph, not a calculator problem. The seat count is a root node. Margin is a leaf. Between them: twelve intermediate calculations, each with conditional logic, each with business rules that don't exist in any training dataset because they're specific to this customer, this contract, this deal.
LLMs process text sequentially. They see tokens. They don't see graphs. Asking an LLM to trace dependency relationships in business math is like asking someone to assemble IKEA furniture by reading the instructions aloud, one word at a time, without looking at the pictures. They'll produce something. It might look like a bookshelf. It will not support books.
A 2024 Deloitte survey found that 47% of enterprise AI users admitted to making at least one major business decision based on hallucinated content (Deloitte, 2024). Nearly half. Not 47% of AI skeptics grumbling about risk. 47% of people who actively use AI in their enterprise, who presumably understand the technology well enough to deploy it, got caught by output that looked right but wasn't.
That's not a user error problem. That's an architecture problem. When the output of a system looks indistinguishable from correct output but is frequently wrong, the system has been designed to produce misplaced trust. The solution isn't better training for users. The solution is a different architecture.
The Architecture That Fixes It
Here's the insight that changes everything: don't ask the AI to do math. Ask it to reference math.
The AI's role is orchestration and presentation, not computation. The number itself comes from the engine. Always. No exceptions.
Build a deterministic calculation engine. Rules-based. Auditable. Traceable. The kind of system your CFO can sleep next to. Give it every business rule, every pricing tier, every discount structure, every escalator formula, every conditional waiver. Make it compute with the rigidity of a tax code interpreter, because that's effectively what it is.
Then give the AI a different job entirely.
The AI's role in this architecture is orchestration and presentation, not computation. It knows which calculation to invoke. It knows how to ask clarifying questions. It knows how to present results in natural language, in tables, in executive summaries. It knows how to explain why the number is what it is. But the number itself comes from the engine. Always. No exceptions.
Think of it as the speechwriter and the accountant. Both essential. Different jobs. The speechwriter makes the quarterly earnings call compelling. The accountant makes the numbers real. You would never ask the speechwriter to produce the numbers. You would never ask the accountant to make them engaging. Same arrangement.
In practice, this means a JSON calculation engine with recursive descent parsing. Pricing tiers are encoded as conditional expressions (IF seat_count exceeds threshold, apply rate X), not prompts. Discount structures are dependency graphs with typed edges (stackable, non-stackable, conditional, absolute). Every intermediate calculation rounds to two decimal places, because that's how money works. Your accountant doesn't carry ten decimal places through a spreadsheet. They round at every step, because every step needs to match what a human would calculate on a 10-key. The rounding strategy is a design decision, not a limitation. When your CFO checks line 47 against their own math, it matches. Every time. A separate validation layer uses a 1e-10 tolerance to catch floating-point drift in comparisons, because IEEE 754 arithmetic occasionally produces 99.999999999 instead of 100.00, and the engine needs to recognize those as equal without silently changing the number.
The AI references this engine's output. It can say "The total contract value is $428,640 over three years, reflecting a 15% volume discount on seats 16 through 50 and a 12% annual escalator on the post-discount base." Every number in that sentence came from the engine. Every word around those numbers came from the AI. Clear division of labor. Clear chain of custody for every dollar figure.
This is not a limitation of the AI. This is what good architecture looks like. The AI is not diminished by not doing math. It's freed to do what it's actually good at: understanding context, generating language, explaining complexity in human terms. Asking it to also be a calculator is like asking your best salesperson to also be the bookkeeper. You're not using a strength. You're creating a liability.
The Verification Layer
Separation isn't enough by itself. You also need verification, because humans are creative and the real world is adversarial to clean architectures.
The verification layer runs as a second pass after the AI produces its output. It checks that the AI didn't smuggle in any numbers of its own. Every financial figure in the document must trace back to a calculator_ref token from the engine. A literal number that doesn't reference the engine gets flagged and rejected. The guardrails also enforce margin and discount limits: is the discount within the allowed range? Does the margin stay above the floor? Is the AI presenting a number the engine actually computed, or did it hallucinate one that looks close enough?
These aren't trust-but-verify checks. These are trust-nothing checks. The engine is deterministic, but the AI is the one assembling the final document, and the AI's job is language, not arithmetic. The verification layer ensures the boundary holds. If a number appears in the output that the engine didn't produce, the system catches it before it reaches the client.
It also catches the Excel problem. Customers live in Excel. They export your pricing to a spreadsheet, adjust numbers, add columns, delete rows, rename cells, and then paste it back. The numbers that come back from that round trip may not match the numbers that left. Delta detection compares every figure against the engine's last known-good state and flags discrepancies. Not silently corrects. Flags. Because silent correction is a lie, and lies in financial documents compound faster than interest.
Beyond Sales
This architecture isn't sales-specific. The same principle applies anywhere a hallucinated number has consequences.
Medical dosing. A drug interaction calculator can't predict what a correct dosage looks like. It has to compute it. The dependency graph includes patient weight, kidney function, concurrent medications, contraindications, and half-life calculations that interact in ways no language model can safely approximate.
Engineering specifications. A structural load calculation for a building isn't a creative writing exercise. The math is either right or the building isn't.
Legal liability caps. Insurance calculations. Financial projections. Revenue recognition. Anywhere the gap between "looks right" and "is right" carries material consequences, the architecture is the same: deterministic engines own facts, probabilistic models own language, and they never switch jobs.
Karpowicz demonstrated in a 2025 paper, using three independent mathematical frameworks, that hallucination is structurally inevitable in current LLM architectures (Karpowicz, 2025). Not a bug to be fixed. Not a problem to be trained away. A mathematical property of how these systems generate output. The implication: if you're waiting for the next model release to make your AI trustworthy with numbers, you'll be waiting through every release forever. The fix isn't a better model. It's a better architecture.
The Uncomfortable Truth
The industry doesn't want to talk about this. Every AI vendor's demo shows the model generating beautiful proposals with impressive numbers. Nobody asks where the numbers came from. Nobody reruns the demo to see if the same numbers appear. Nobody traces the dependency chain from seat count to total contract value.
Plausible is the most dangerous word in finance.
It's a magic show. And the audience doesn't check behind the curtain because the show is so compelling. The proposal looks professional. The formatting is perfect. The dollar signs are in the right places. Plausible is the most dangerous word in finance.
You don't ask your marketing team to do your taxes. Not a knock on marketing. They're great at what they do. But what they do is not accounting. You give them the numbers accounting produced and let them make it look good. AI is marketing. The calculation engine is accounting. Let them do their jobs.
The architecture that separates language from math isn't a workaround. It's not a temporary fix until models get better at arithmetic. It's the recognition that two fundamentally different kinds of computation exist, and mixing them produces output that's worse than either one alone. Deterministic engines give you numbers you can trace. Probabilistic models give you language that connects. Together, with clear boundaries, they build things people trust with real money.
Apart, or worse, merged into a single prompt that does both? Three answers, three restaurants, and a signed contract with the wrong total.
The takeaway: Deterministic engines own facts, probabilistic models own language, and they never switch jobs: that's the architecture that makes AI-generated numbers trustworthy.
References
1. npj Digital Medicine. (2025). Multi-model assurance analysis showing large language models are highly vulnerable to adversarial hallucination attacks during clinical decision support. Hallucination rates of 50-82% across models; GPT-4o at 53%, reducible to 23% with structured mitigation prompts. 2. MIT. (2025). Research on AI confidence calibration. Models 34% more likely to use high-confidence language when generating incorrect information. 3. Deloitte. (2024). State of Generative AI in the Enterprise survey. 47% of enterprise AI users made at least one major business decision based on hallucinated content. 4. Karpowicz, B. (2025). Mathematical impossibility of eliminating hallucinations in LLM architectures, demonstrated across three independent proof frameworks (auction theory, proper scoring theory, log-sum-exp analysis for transformer architectures). 5. Nielsen Norman Group. Usability Heuristic #9: Help Users Recognize, Diagnose, and Recover from Errors. Referenced for the broader principle that system transparency builds user trust. 6. OpenAI. (2025). Why Language Models Hallucinate. Acknowledgment that hallucinations remain a fundamental challenge driven by training incentives that reward confident guessing over calibrated uncertainty.
This paper is part of The Architecture Series by Ed, Founder & CEO of DealiOS. For the weekly series and more technical deep-dives, visit dealios.app/library.
See how these principles come together. Explore the platform →