The idea behind it
A pilot answers whether an AI feature can work on real cases with a small group of users. Production asks more: whether the feature keeps working as volume grows and inputs vary, whether it fails safely, whether it is secure against people who try to misuse it, whether it costs what the business case said, and whether someone owns it when it breaks. A stage gate puts those questions in one meeting, with the evidence on the table, and records a decision that the accountable people sign.
This gate has 31 criteria in ten sections, and 19 of them are must-haves. The security section follows the OWASP Top 10 for LLM Applications 2025, which lists prompt injection first, including indirect injection through documents, emails or tickets the model reads, followed by sensitive information disclosure; it also covers excessive agency, improper output handling and unbounded consumption. The evaluation and monitoring sections correspond to the Measure and Manage functions of the NIST AI Risk Management Framework. The decision rule has three outcomes. Any must-have not met stops the feature. Any remaining gap gives a conditional go, with each condition owned and dated. Only a clean result is a go.
What's inside
- A key figures table: each metric's baseline, pilot result and production threshold, and whether it meets the threshold.
- 31 criteria in ten sections: measured value, evaluation, failure handling, data and privacy, security, cost, monitoring, owner and support, users, and rollback. 19 are must-haves.
- Security criteria for prompt injection, sensitive information disclosure, excessive agency, output handling and unbounded consumption, from the OWASP Top 10 for LLM Applications 2025.
- A decision rule: stop if a must-have is not met, conditional go if any gap remains, go if every must-have is met.
- A conditions table with owner and due date, and a sign-off block for the business, technology, security, privacy and risk owners.
- A worked example for a fictional company's IT service desk pilot that returns Conditional go with three conditions. The Excel version calculates the decision and flags gaps without a condition.
Format: PDF (6 pages) and Excel workbook.
A look inside
Pages 1 and 2 of 6 of the PDF. Select an image to enlarge it.
Who uses it
- CIOs, CTOs and heads of AI, to decide whether a pilot moves to production on written criteria.
- Security architects, to test prompt injection, data leakage and tool permissions before wider rollout.
- Privacy and risk leads, to confirm the data review, vendor terms and regulatory classification.
- Business owners, to accept the feature with its conditions and support arrangements.
When to use it
- A pilot has ended and the team is asking to roll the feature out to more users.
- A vendor's AI feature is being switched on for the whole company after a trial with one team.
- An agent will be given access to tools or systems it did not have in the pilot.
- A feature passed a pilot but its cost at production volume has not been measured.
How to use it
Agree the must-haves before the pilot starts, then hold the gate when the pilot ends and before the feature reaches more users.
- Agree the gate's members and the must-have criteria before the pilot starts, so the pilot collects the evidence they need.
- Two weeks before the gate, the pilot team fills in the key figures and the evidence for each criterion.
- At the gate, set a result for each criterion from the evidence: Met, Partly met, Not met or Not applicable. A criterion with no evidence is Not met.
- Apply the decision rule, and write a condition with an owner and a due date for each gap.
- Have the business, technology, security, privacy and risk owners sign, and record any disagreement.
- Track the conditions to closure; a condition that misses its due date returns the feature to the gate.
You leave with: A signed go, conditional go or stop decision for the feature, with the evidence for each criterion, the key figures against their thresholds, and dated conditions with owners.
Where it goes wrong
- Setting the thresholds after seeing the pilot results, so every result passes.
- Testing prompt injection only through the chat box and not through the documents, emails or tickets the AI reads.
- Treating the pilot team's summary as evidence instead of the test results and figures behind it.
- Granting a conditional go without due dates, so the conditions stay open after the rollout.
Get the checklist
It is free. Enter your work email and we send the download link.
Questions
What criteria should an AI pilot meet before production?
Measured value against a baseline, evaluation results that meet thresholds set in advance, safe failure handling and a human fallback, data and privacy review, security testing for prompt injection and data leakage, cost per task at production volume, monitoring, a named owner and support, trained users, and a tested rollback.
What is a conditional go?
A decision to proceed with named gaps still open. In this gate it applies when no must-have criterion is Not met but at least one criterion is Partly met or Not met. Each gap gets a condition, an owner and a due date, and the rollout plan says which conditions close before which step.
How do you test an LLM feature for prompt injection?
Write test cases that place instructions in every input the model reads, not only the user's prompt: documents, emails, tickets and web pages. Check whether the output or any tool call changes. OWASP lists prompt injection, direct and indirect, as LLM01 in its Top 10 for LLM Applications 2025.