A demo and a decision are different commitments
A demo proves that a model can produce an answer. It proves nothing about whether the answer will be used. Those are separate problems, and the second one is the expensive one.
When a pilot ends, somebody has to take the output and act on it. If that person was not part of designing the thing, they will do what any sensible operator does with an unfamiliar number: check it by hand. Once the output is being checked by hand, the work has not been automated. It has been duplicated. Six months later the pilot is quietly dropped and the conclusion recorded is that the technology was not ready.
The technology was ready. The handover was never designed.
The four conditions
These are the four things I now insist on before calling anything production. They are not technical requirements. Three of them are organisational, which is exactly why they get skipped.
1. The decision has a name attached to it
Not the model. The decision. One person is accountable for the outcome of what the agent decides, and they know it, and they agreed to it. If the answer to "who owns this" is "the analytics team", the agent is a report with extra steps. Analytics teams do not own operating decisions and should not pretend to.
The useful side effect is that an owner with real accountability will tell you very quickly which of the agent's decisions they do not trust, and why. That list is worth more than any accuracy metric.
2. It runs on the operating calendar
Anything that runs when someone remembers to run it is a pilot, regardless of how good it is. Production means the agent fires on the same rhythm the business already has, and its output arrives before the meeting where the decision used to be argued about. If the operating cycle is weekly, the agent is weekly. If it misses a week, somebody notices without being told.
This is also the cheapest test of whether a project is real. Ask when it runs. A vague answer means it does not.
3. Every decision is written down before it is executed
Each decision the agent takes gets logged with its inputs, its reasoning and the time it happened, and the log is readable by someone who is not an engineer. This is not a compliance formality. It is the only way to answer the question that always comes, usually three months in and usually from a country manager: why did it do that to my store?
Without a record you have two bad options: defend the agent on faith, or turn it off. I have seen both. Turning it off wins.
4. The output lands in a number the business already reports
If the effect of the agent only shows up in a dashboard the agent's own team built, it does not exist as far as the organisation is concerned. The result has to arrive in the same figures that were being reviewed every month before anyone had heard of the project, produced by the same process, reconciled the same way.
This is the condition that gets the most resistance, because it removes the possibility of a flattering private scoreboard.
None of these four require a better model. All four require someone senior enough to make organisational commitments, which is the actual scarce resource in corporate AI.
What I would look at first in your operation
Find the recurring decision that is still made by one person in a spreadsheet. Every operation has several. Then ask three questions about it.
- What happens when that person is on holiday? If the answer is "it waits" or "someone guesses", the decision is already a single point of failure and automating it is defensible on those grounds alone.
- Could you reconstruct why last month's version came out the way it did? If not, you have no baseline, and you will not be able to prove the agent improved anything. Fix the record before building the agent.
- Is the input data something the company already governs? If the spreadsheet is fed by a file someone emails, the agent will inherit that fragility and get blamed for it.
The decisions that pass all three are where I would start. They are usually unglamorous and they are usually worth a lot, because work that is done by hand every month has been quietly paying a tax for years.
The part nobody wants to hear
The sequence matters more than the ambition. Instrument the process, find where the capital is misplaced, automate the decision, then prove the result against a control group. Attempting step three before steps one and two is the single most common way this work fails, and it fails slowly enough that nobody is held responsible for it.
The unglamorous prerequisites are covered in note 03. The proving part is note 02, and it is the one that decides whether anyone believes you.