Notebook / Applied AI / 13.09.2026
Human review is an operation.
Putting a person “in the loop” is not enough. Define what they receive, what they can decide, how much context they need and how their decision returns to the system.
An approve button is not oversight
When a model interprets a document, proposes a classification or drafts a reply, an interface may show “accept” and “reject”. That only creates two buttons. The reviewer still needs to know which source produced the proposal, which rule runs next and what a wrong decision would affect.
NIST's AI RMF asks organisations to define and document roles, responsibilities and human oversight processes. It also calls for evaluation in conditions close to deployment and monitoring while systems operate. In design terms, review needs context and a stable place in the workflow.
Five parts of useful review
- Owner. A specific role, rather than “the team”, receives the exception.
- Context. Source, relevant fields, proposal and reason for uncertainty appear together.
- Action. The reviewer can correct, refer, request information or stop the route.
- State. Pending, reviewed, blocked and closed stay distinct; missing input never becomes approval.
- Evidence. The system records what changed, when and under which rule, while minimising data.
Design the stop rule too
Not every case should escalate, and not every automation should continue. Scope needs thresholds: poor input quality, missing permissions, sensitive data, format changes or decisions with significant impact. At those boundaries, the right behaviour may be to stop and request review.
Palantir's AI FDE best-practice documentation recommends verifying generated resources before production, using representative data and limiting tools and context. These are concrete controls that remain useful beyond a specific platform.
What to measure before expanding
Track how many cases reach review, how long decisions take, which error types are corrected and how many alerts add no value. These measures do not prove the system is appropriate, but they show whether it moves work forward or creates a new queue.
The aim is to give a person more judgement and better context, rather than make them click through every machine output. Exception design is part of the product.