Should a human review AI output before it acts, or after?

Reversal cost decides the placement

Sort every AI-driven action by what it takes to undo. An internal note written to a job record costs nothing to correct. A text message to a customer cannot be recalled. A status change that triggers an invoice takes an accounting adjustment. Put the review gate immediately before the first step where reversal becomes expensive, and nowhere earlier.

• Internal summaries, tags and briefs. No gate. Review by sampling after the fact.
• Drafts that reach a customer. Gate before sending, always, without exception for volume.
• Evaluation that touches a person's work. A human sets the standards and makes every coaching decision. The system supplies the evidence and the consistency - see rubric-based evaluation.
• Anything financial. Gate, and log who approved it.

Rubber-stamping is a real and predictable failure

Ask a person to approve two hundred items a day and by the second week they are clicking approve. The review exists on the org chart and not in reality, and because it exists on paper, nobody builds the monitoring that would have caught the problem.

The rule of thumb: if a reviewer must handle more items than they can genuinely read, you have not built a control. Either reduce the volume reaching them or replace gating with sampling.

Sampling usually beats gating

For high-volume, low-reversal-cost output, a better design reviews a random sample plus every item flagged as unusual - low confidence, unusual length, contradicted by another system, outside a normal range. That gives you a measured quality rate rather than a false sense of complete oversight, and it concentrates human attention where it changes something.

Track the correction rate from that sample over time. It is the most honest live quality metric you will have, and it is what tells you whether a model change or a prompt change actually helped.

Make the reviewer's job mechanically easy

Reviewers are fast when the evidence sits next to the output: the call excerpt that produced a score, the fields that drove a recommendation, the record that was matched. They are slow when they must open three systems to check one item, and slow review turns into no review.

Corrections should also go somewhere. A one-click correction that feeds a stored example set turns review time into a permanent improvement in the evaluation data rather than a repeated cost. Building that loop is a normal part of custom AI development, not an advanced feature.