Enterprise AI adoption & strategy

From Pilot to Production: A 90-Day Gate Review Framework for Department-Level AI Projects

· By AIHQ Team

Colleagues reviewing a one-page printed scorecard at a department AI pilot gate review in a corporate meeting room

Most departmental AI pilots stall because nobody scheduled the decision to continue

Three pilots start in Q1. By Q3, all three are still described as "in progress." Two have quietly lost their sponsor to a reorganisation. One is used daily by four people. Nobody on the executive committee can say whether any of them should get more money, and the finance team is carrying three line items it cannot resolve.

That outcome is rarely a technology failure. It is a decision-timing failure. Departments launch pilots with an implicit assumption that momentum will carry them forward, and they never set the date on which someone has to say yes or no.

A gate review fixes that. It is a scheduled, short, criteria-driven conversation at fixed points — day 30, day 60, day 90 — where a pilot either passes into the next stage, gets a defined repair plan, or stops. The CFO gets a capital answer, the board gets visibility, and the department lead gets permission to kill something without it being read as a personal setback.

This framework is written to be run by a department head without a central PMO. It assumes a budget in the range of RM40,000 to RM150,000 for a departmental pilot in Malaysia, a named business owner, and a working prototype or tool subscription already in place.

Why pilots fail on time, not on technology

Look at where departmental AI pilots die and the pattern is consistent: not at the demo, but at the handover between the pilot team and the operational team that has to run it.

The pilot team has a builder's incentive — keep improving. The operational team has a workload incentive — this is extra work with no reduced headcount to show for it. In between sits a governance gap: no one is empowered to demand evidence, and no one is required to supply it.

The consequence is an expensive holding pattern. Three pilots at RM60,000 each is RM180,000 of capital sitting in a state that is neither approved nor closed. In an organisation with a 30-pilot portfolio, that number becomes material to the annual budget conversation.

The other hidden cost is attention. Every stalled pilot keeps a senior sponsor mentally occupied and a slice of IT capacity reserved. Gates release that capacity deliberately, whether the outcome is go or stop.

The three gates at a glance

Each gate has a single question, a small number of pass criteria, and three possible verdicts. Keep the meeting to 45 minutes and the evidence to one page.

Gate Timing The question it answers Default verdict if no evidence
Gate 1 — Specificity Day 30 Is this one workflow, one owner, one measurable output? Park
Gate 2 — Traction Day 60 Are real users using it, and is access and risk cleared? Fix with named actions
Gate 3 — Production readiness Day 90 Can this run without the pilot team, at a known cost? Kill or re-scope

Gate 1 (day 30): use-case specificity and workflow ownership

Pass criteria

  1. The use case is described in one sentence naming a workflow, not a technology. "Draft first-pass responses to motor claims enquiries" passes. "Explore AI in operations" does not.
  2. A named workflow owner exists — a specific manager accountable for the process today, not a project coordinator.
  3. A baseline number exists, measured before the pilot: cycle time, handling volume, rework rate or error rate.
  4. The data the pilot needs has been identified by name and location, and the person who controls it has agreed to share it.

Fail signals: the baseline is "we'll measure later"; the scope grows to cover three departments; the data owner has not been told the pilot exists.

What fail looks like in practice: a shared-services team spends a month building a document-summarisation assistant before anyone confirms which document set legal will permit. The pilot is technically working and commercially dead.

Gate 2 (day 60): data access, adoption signal and risk clearance

Pass criteria

  1. Real users — not the pilot team — have run the workflow at least weekly for three consecutive weeks. Aim for a floor of 60% of the intended user group having tried it at least twice.
  2. The workflow has produced at least 20 completed outputs that went into real work, not test cases.
  3. Data handling has been reviewed against your organisation's own policy and Malaysia's Personal Data Protection Act 2010 (PDPA) where personal data is involved: what is stored, where, for how long and who can retrieve it.
  4. Confidential-material guardrails are documented and tested, including what staff must not paste into a general-purpose tool.
  5. A human review step exists in the workflow and someone is named as the accountable reviewer of AI-assisted output.

Fail signals: usage is concentrated in one enthusiastic individual; the risk or legal function hears about the pilot at this meeting for the first time; the pilot has no documented answer to "what happens to the data."

Data privacy is not a compliance ritual here. PDPA obligations around notice, purpose limitation and retention apply to AI-assisted processing just as they do to any other processing, and the department that owns the workflow usually owns that obligation. If your organisation has not settled the usage boundaries yet, run a short responsible AI and governance workshop before Gate 2 rather than after.

Gate 3 (day 90): production readiness and cost transparency

Pass criteria

  1. Run cost is known in MYR per month at current volume, including licences, usage-based model consumption and any integration support.
  2. A named operational owner is in place outside the pilot team, with the workflow written into their job scope.
  3. The workflow runs for two consecutive weeks without the pilot lead being present.
  4. A measurable change against the Gate 1 baseline is documented — direction and magnitude, with the measurement method stated. A 22% reduction in first-draft handling time with a defined sample is evidence. "Feels much faster" is not.
  5. A decision is recorded on build-versus-buy: does this stay on the off-the-shelf tool, or does it need custom AI workflow and implementation support?

Fail signals: the monthly cost is unknown or estimated at zero; the only person who can operate the workflow is the person who built it; the integration relies on a personal account rather than a company one.

The one-page gate scorecard a department lead can run

Score each criterion 0, 1 or 2. Zero means absent, 1 means partial or undocumented, 2 means evidenced in writing.

Gate 1 — maximum 8 points. Pass at 6 or above with no zero on criterion 2 (workflow owner).

  • C1 Use case names one workflow — 0/1/2
  • C2 Named workflow owner — 0/1/2
  • C3 Baseline metric measured before pilot — 0/1/2
  • C4 Data source identified and access agreed — 0/1/2

Gate 2 — maximum 10 points. Pass at 7 or above with no zero on C3 (data handling) or C5 (accountable reviewer).

  • C1 Weekly usage by real users for three weeks — 0/1/2
  • C2 At least 20 outputs used in real work — 0/1/2
  • C3 Data handling reviewed against policy and PDPA — 0/1/2
  • C4 Confidential-data guardrails documented — 0/1/2
  • C5 Named accountable reviewer for AI-assisted output — 0/1/2

Gate 3 — maximum 10 points. Pass at 8 or above with a 2 on C1 (known monthly cost).

Hand-drawn one-page gate review scorecard on white paper with marker boxes, arrows and highlighter accents

One page, three gates, three verdicts: pass, fix or stop.

  • C1 Monthly run cost stated in MYR — 0/1/2
  • C2 Operational owner named outside pilot team — 0/1/2
  • C3 Two weeks running without pilot lead — 0/1/2
  • C4 Measured change versus baseline — 0/1/2
  • C5 Build-versus-buy decision recorded — 0/1/2

The scoring is deliberately coarse. The purpose is not precision — it is to force a written answer where currently there is a verbal impression. A department lead who cannot score a criterion has usually discovered the real blocker.

Fix, park or kill: what to do when a pilot fails a gate

Boards lose more value from pilots that neither succeed nor stop than from pilots that fail cleanly. Three verdicts, each with a defined next step:

Fix. Use when the gap is specific, bounded and correctable within one gate cycle. A missing baseline, an unnamed owner, an untested guardrail. Fix comes with a named person, a written action list of no more than four items, and a re-review date inside 21 days. One fix is normal. Two consecutive fixes at the same gate is a signal to escalate.

Park. Use when the use case is credible but the preconditions do not exist yet — the data is not ready, the upstream system is being replaced, or the regulatory position is unsettled. Parking is not failure; it is sequencing. Record what must become true before restart, set a review date no longer than two quarters out, and release the team. Parked pilots must appear on a register, or they quietly become zombie pilots.

Kill. Use when two conditions hold: no measurable movement against baseline after a proper trial, and no named operational owner willing to take the workflow on. Kill the pilot, document the learning in two paragraphs, and return the budget. The second condition matters more than the first — a workflow with no operational owner will not survive, regardless of how good the demo is.

Designing the gates into the funding decision, not after it

Gates only work when they are attached to money before the pilot starts. Three practical mechanisms:

Release budget in tranches. Fund day 1–30 fully, then release the next tranche on Gate 1 pass. Departments respond to payment schedules far more reliably than to governance memos.

Write the gates into the business case. The one-page pilot approval should state the three gate dates and the named decision panel — typically the department head, a finance representative and the risk or compliance lead.

Publish a portfolio view quarterly. A simple register of active, parked and killed pilots with spend to date gives the board the one thing it usually lacks: a count and a cost. That register is also the natural input to next year's AI implementation cost model for finance and department heads.

If your organisation is also negotiating supplier contracts alongside these decisions, the AI procurement tender specification framework covers exit terms and lock-in clauses that affect Gate 3 cost transparency.

Governance layers that sit above the gates

Departmental gates do not replace organisational governance — they feed it. Three questions belong at the executive layer, not in the department review.

Who decides when a pilot becomes a shared service? When three departments build near-identical document assistants, the fourth approval becomes a platform question. Decide in advance which patterns get centralised.

Who owns the portfolio risk register? Individual pilots clear their own risk at Gate 2; the aggregate picture — how many workflows now touch personal data, which vendors hold what — belongs to a single accountable executive.

Who signs off when the workflow affects customers or the public? Any pilot touching customer communications, pricing, eligibility or regulated advice needs a review threshold above department level. Regulated and public-sector organisations usually need a stricter control set; the AI agent governance framework for public sector and GLCs sets out accountability owners and risk tiers for that case.

When the pilot is a read-only assistant, gates are simple. When it takes actions — sending, filing, approving — the controls framework needs to be in place before Gate 3, not after. That distinction is worth making explicitly in the business case.

Frequently asked questions

How long should a departmental AI pilot run? Ninety days is a practical default for a single-workflow pilot with an existing tool. Pilots that need custom integration or data preparation often need 120–150 days — but keep the gate dates, and add a fourth gate rather than dropping the discipline.

Who should sit on the gate panel? Three people: the department head as chair, a finance representative who can release the next tranche, and a risk, legal or compliance representative. Keep it to three so the meeting happens. Add IT as an attendee rather than a decision-maker unless the pilot touches core systems.

Is a failed gate a career risk for the department lead? Only if the organisation lets it be. Publish the rule in advance: a documented kill at Gate 3 is a successful use of the process, not a failed project. Organisations that punish kills get zombie pilots instead — which cost more and surface later.

What if the pilot passes every gate but finance will not fund production? That is a portfolio decision, not a gate failure. The pilot's job was to produce evidence cheaply. If the evidence is good and funding is unavailable, park the pilot with a defined restart trigger rather than extending it indefinitely at pilot scale.

How does this work when several departments run pilots at once? Use one scorecard and one register across all of them. The value at portfolio level is comparison: which department reached Gate 2 fastest, which pattern repeated, where spend is concentrated. A departmental programme cannot answer those questions alone.

Start with the dates, not the technology

A pilot without a scheduled decision date is a subscription, not a project. Before the next departmental AI initiative is approved, write three dates into the business case — day 30, day 60, day 90 — attach a named owner to each gate, and agree in advance that a documented kill is a legitimate outcome.

The pilots that reach production are rarely the most technically ambitious. They are the ones where someone was required to show evidence on a fixed date, and someone else was empowered to act on it.

FAQ

What should a day 30 gate review actually cover?

One use case, one workflow owner, one measured baseline, and confirmed access to the named data source. If any of those four is missing, the pilot should be parked and re-scoped rather than funded into the next stage.

How do I measure adoption at the day 60 gate?

Use two numbers: the share of intended users who have run the workflow at least twice, and the count of outputs that went into real work rather than testing. Usage concentrated in one enthusiast is a warning signal, not adoption.

What PDPA considerations apply to a departmental AI pilot?

Where personal data is processed, PDPA obligations around notice, purpose limitation, retention and access still apply — the tool being AI-assisted does not change them. Document what data is stored, where, for how long and who can retrieve it before Gate 2.

Does every pilot need a custom solution at Gate 3?

No. Most workflows stay on an off-the-shelf tool. The Gate 3 build-versus-buy decision simply records the answer in writing, with a cost basis, so the choice is deliberate rather than inherited.

How much should a departmental AI pilot budget be?

For a single-workflow pilot with an existing tool, RM40,000 to RM150,000 is a realistic range in Malaysia depending on integration and training effort. If the pilot needs custom development, budget it separately and expect a longer gate sequence.

← Back to all articles