Your AI Pilot Needs Kill Criteria, Written Down, Before You Build
McKinsey’s estimate that roughly 90 percent of function-specific AI use cases never leave the pilot phase gets quoted as a tragedy. Having watched a fair number of these pilots up close, I read it differently: most of those pilots didn’t fail. They just never ended.
A pilot without pre-agreed kill criteria cannot fail. It also cannot succeed. It produces demos, enthusiasm, a steering-committee slide, and then a quiet fade as the sponsor moves on. Six months later someone asks what happened to the invoice-coding project, and the honest answer is that nothing happened to it. It is still “in pilot.” The 90 percent graveyard is mostly filled with projects that were never given a way to die — and therefore never forced to earn the right to live.
The fix is embarrassingly cheap. Before anyone writes a prompt, write one page: the workflow and its single accountable owner, the measured baseline, the success threshold that triggers a scale decision, the eval set that will measure it, the permission line (what the system may do without a human), and — the two fields every team resists — the kill criteria and the calendar date of the decision meeting.
Teams resist kill criteria because writing them feels like planning for failure. It is the opposite. A pilot that dies on schedule, with a documented reason, is cheap intelligence: you learned the workflow isn’t verifiable enough, or the data isn’t there, or the error tolerance is too tight, and you moved your budget to the next candidate. A pilot that fades without a verdict is expensive ignorance — you paid for it and learned nothing, because nothing was ever measured against anything.
There is also a cultural effect I did not expect when I started recommending this. Kill criteria make teams bolder, not more cautious. When everyone knows the pilot ends on a date, with a number, people stop protecting it and start testing it. The demo theater stops. The domain experts get blunt. That bluntness is where the real requirements surface.
If you want the one-page template, it’s free in the book’s FDE Toolkit. Steal it. The only part you must not skip is the part that feels most uncomfortable: deciding, in advance, what evidence would make you stop.