Notebook / AI applied to work
From an AI pilot to a process your team can operate.
A convincing answer in a demonstration does not explain who will review the next document, how uncertainty will be resolved or what happens when the destination fails. A useful pilot tests that whole workflow.
By Kenea · Published and reviewed: .
Interpretation needs review.
- Document
- Criterion
- Review
Start with a small output you can verify
Replace “add AI to operations” with a task: prepare a draft request from a document and show the source of each field. In this hypothetical example, a person decides whether the draft can become a record. Orders are not sent and commitments are not accepted automatically.
Write down what falls outside the scope. An unreadable document, two possible customers or an unknown reference may need review. If rules can handle the task, a model may be unnecessary. The AI or automation guide helps distinguish those cases.
Prepare the data and reserve an evaluation sample
Use authorised documents and only the data you need. Establish who may process it, which service will receive it and how long it will be retained. Include common formats and difficult cases; selecting only clean documents would give an incomplete picture of the work.
Separate the examples used to adjust the solution from those used to evaluate it. For the latter, prepare an expected answer checked by someone who understands the process. Keep the configuration version and record changes between trials.
The NIST AI Risk Management Framework is a voluntary reference for incorporating risk management into the design, use and evaluation of AI systems. It is not a certification of this pilot and does not replace its tests.
Make human review a real task
“Someone will review it” leaves too many questions open. That person needs to see the document, the proposed field and the reason for uncertainty. They must be able to correct, reject or leave an item pending without the system treating those actions as approval.
Assign an owner and backup, handling time and priority. Measure the effort of reviewing, not just the model's response time. In our example, a draft queue that grows every day would reveal a capacity problem even if extraction looked fast.
Test exceptions before connecting actions
Include a duplicate document, missing field, unavailable destination and incomplete model response. Each should leave a visible state and an available next action. Without confirmation from the receiving system, do not mark the work complete or repeat a write without checking for duplicates.
Documents may also contain instructions intended to manipulate the model. OWASP describes this indirect prompt injection and recommends limited permissions and human controls for high-risk operations. Source: OWASP, prompt injection prevention. In this proposed pilot, reading a file does not grant authority to execute its instructions.
Agree on acceptance and stopping conditions
This checklist is Kenea's proposal for the example, not a universal standard. Agree on thresholds with the process owner before running the sample:
| Dimension | Evidence needed |
|---|---|
| Quality | Correct fields and errors grouped by severity, beyond an overall average. |
| Review | Human handling time and backlog, including rejections and corrections. |
| Control | No writes outside the scope; an owner for each exception. |
| Recovery | A tested procedure to stop and resume without duplication. |
| Cost | Service usage, supervision and estimated maintenance. |
A data leak or unauthorised action would require stopping the trial and investigating. Other findings may justify adjusting the scope. Passing a small sample does not demonstrate that future errors are impossible: document the cases tested and those still outstanding.
Prepare for the following day
Before extending the solution, agree on who handles incidents, how changes to the model or software are reviewed and what triggers another evaluation. The decision may be to continue, narrow the scope or abandon that approach. All three are valid when supported by evidence.
The free visual assessment helps map the process. A separate professional assessment can define the data, dependencies and tests before development.
Sources and editorial approach
NIST AI RMF and OWASP, linked beside the relevant statements, consulted on 9 September 2026. The workflow and checklist are Kenea's original proposals. They do not describe a deployed client project or guarantee accuracy, savings or complete security.