From an AI demo to a process your team can use

A demonstration often uses a selected file, a carefully worded question and someone ready to interpret the answer. Everyday work brings different conditions: incomplete documents, unfamiliar formats, permissions and people who need to continue their task when a model fails.

The move to production begins when those conditions become part of the design.

One task, from start to finish

Suppose we want to help a team review supplier documents. They receive files, check specific fields and record what is missing. This is an illustrative example, not a client result.

The initial scope could cover one document type. The application receives the file, suggests several values and shows where they appear. A person confirms or corrects them. Only then are they saved to the supplier record.

This provides a clear unit of work. It helps establish what AI should do and what should remain under human control. It also avoids making the first delivery dependent on replacing every tool the company uses.

What deserves a test

NIST’s Generative AI Profile places evaluation, documented risks and oversight within system management. It is a technical reference for organising the work; completing a checklist is not a certification. Source: NIST AI 600-1.

For our example, we would prepare a representative sample of documents, with permission to use them, and define the expected results before evaluating the tool.

Situation	What to check
A complete, typical document	Suggested fields match the original
Missing or ambiguous information	It is flagged for review rather than invented
A user without access	Neither the file nor its extracted contents can be viewed
An unavailable AI service	The team can continue or recover the task without duplicates
A human correction	The final value and the necessary change history are retained

Acceptance thresholds depend on the use and its consequences. An accuracy figure is insufficient without understanding the errors it contains and who will review them.

Measure the complete task

Fast extraction can create more work for the person making corrections. Measure the complete task time, errors found and outstanding reviews, comparing them with a sample of the current process.

Also examine the cost of processing a document and what happens with longer files or peaks in usage. Model usage is one part of the total cost; maintaining connections, permissions and traceability requires work too.

What should be included at handover

An initial version needs an owner, acceptance criteria and an incident procedure. Its documentation should make the tool’s behaviour and limits clear, including how to return to the previous process if necessary.

At Kenea, we use this conversation to scope the development: what the application receives, what it suggests, who validates it and where it saves the result. That map supports a concrete proposal and a decision about whether the project is worth building.