Can the AI make things up about our calls and our numbers?
Yes, and the failure mode is specific. Models rarely invent an entire call. They invent connective tissue — a reason the customer declined that was never stated, a cause for a revenue change the model cannot see, a confident figure in a summary. The defenses are structural: make the model quote source text, compute every number in code rather than in prose, and never let a model state a figure it was not given.
Two different kinds of wrong
Extraction errors are when the model misreads what is there — it hears "no" as "know," or tags a call as booked when the customer said they would call back. These are annoying and mostly detectable, because the source text is right next to the claim.
Explanation errors are the dangerous ones. Asked why bookings fell, a model will produce a fluent, plausible reason whether or not the data supports it. Nothing in the output signals that the reasoning was manufactured. Fluency is not evidence.
Numbers should never come out of a language model
This is the single highest-value rule in operational AI. Every figure in a report — call counts, booking rate, revenue by source, week-over-week change — should be computed deterministically in a query, then handed to the model as a value it is allowed to describe but not to derive.
A model that is asked to add, divide or compare will sometimes do it correctly and will occasionally not, and the wrong answer will look exactly like the right one. Our daily intelligence briefs work this way deliberately: the arithmetic happens in code, the model writes the sentence around it.
Force citation, then drop what cannot be cited
For anything derived from a transcript, require the model to return the span it relied on. Then validate that the span actually appears in the source. A claim with no verifiable span gets dropped rather than displayed.
This one control converts a large class of soft errors into a visible gap, and gaps are much easier to manage than confident fabrications. It is also what makes rubric scoring reviewable instead of mysterious.
Where you should still expect disagreement
Even with citations and computed numbers, subjective judgments will vary at the margins — whether a rep showed enough empathy, whether an objection was really handled. Two experienced managers would disagree about the same calls.
The honest position is that these systems reduce and localize error rather than eliminate it. Design for the error you will still have: sampling, dispute paths and version stamping, so a wrong output is caught and corrected rather than quietly acted on.
Topics: hallucination · accuracy · reporting · controls
Have a version of this question about your own business?
The useful answer usually depends on which systems you run and how they're connected. That's a conversation, not a blog post.