How do I know an AI call score is based on something that was actually said?
Require evidence. A defensible evaluation shows, for every criterion, the specific transcript lines that drove the result — quoted, timestamped, and clickable back to the audio. If a score cannot point at text, treat it as an opinion rather than a finding. Evidence links are also what make coaching work, because the manager is discussing a specific moment in a specific call instead of defending a number.
An unsupported score does not survive contact with a rep
The first time a CSR is told they scored poorly on "discovery," they will ask what they did wrong. If the answer is a number, the conversation becomes an argument about the software. If the answer is three quoted lines and a timestamp, the conversation becomes about the call.
That is the whole design goal. Rubric evaluation should apply your standards consistently and then show its work, so the human coaching decision rests on something both people can look at.
What an evidence-linked record contains
- The criterion as written in your rubric, in your language.
- The verdict — met, not met, partial — and the weight it carries.
- The quoted span from the transcript, verified to exist in the source.
- A timestamp that jumps to that point in the audio.
- Version stamps for the rubric and the model that produced the result.
- Reviewer state — unreviewed, confirmed by a human, or overridden with a reason.
Evidence lets you audit the rubric, not just the rep
This is the benefit nobody anticipates. When you read the quoted spans and agree the model found the right moment but still disagree with the verdict, the problem is your rubric wording, not the AI.
That happens constantly with criteria like "created urgency" or "built rapport," which mean different things to different managers. Seeing the evidence forces you to define them. Most teams end up with a sharper standard than they started with, which improves human review too.
A related tell: when the cited spans for one criterion come from wildly different parts of the call every time, that criterion is probably measuring something too diffuse to score. Split it into two concrete behaviors you could point at in a transcript, and most of the disagreement resolves itself.
A spot-check protocol that costs almost nothing
Pull a handful of scored calls each week, read the evidence, and record whether you agree. Track the disagreement rate by criterion rather than overall. A criterion with persistent disagreement is either badly worded or badly suited to automated evaluation, and both are fixable.
That running number is the honest measure of whether the system is working — far more useful than any accuracy claim a vendor could make about it. It also feeds directly into how coaching sessions get prioritized.
Topics: explainability · rubrics · evidence · coaching
Have a version of this question about your own business?
The useful answer usually depends on which systems you run and how they're connected. That's a conversation, not a blog post.