How to Set Up an AI Centre of Excellence: A Step-by-Step Guide

Why the CoE decision lands on the CFO's desk first

A group finance director we work with had 14 separate AI subscriptions running across five departments by month six. Nobody could say what the annual run rate was, which teams were actually using the tools, or what the retained benefit looked like. The licences were small. The unmanaged pattern was not.

That is the situation that usually triggers a Centre of Excellence conversation. Not a strategy retreat — a spend review.

So the useful CFO framing is blunt: an AI Centre of Excellence is a cost and control structure with a mandate. Team charts and aspirational charters come later. If you cannot describe what the CoE will own financially — licences, vendor contracts, training budget, pilot funding, measurement — you are designing a committee, not a capability.

This guide compares three operating models head to head, gives you indicative cost bands in MYR, and lays out a 12-week rollout with named owners.

What a CoE actually owns (and what it should not)

A CoE that tries to own everything becomes a bottleneck. A CoE that owns nothing becomes a newsletter.

It should own:

• Consolidated AI tooling spend and vendor relationships
• The use-case intake and prioritisation process, with scored criteria
• Role-based capability building — including your AI training programmes budget line
• Data handling rules in practice, not just on paper
• Measurement: adoption, usage, cycle time, error rates

It should not own:

• Every AI project. Business units run their own pilots; the CoE sets standards and reviews gates.
• Hiring decisions outside its core team
• Final budget authority below an agreed threshold — that sits with finance

As a rule of thumb for finance leaders, set the CoE's discretionary spend threshold at a level you can delegate without a second approval. In Malaysian mid-market and GLC environments, that is often RM50,000 per initiative, with anything above routed through the usual capex or opex committee.

Three operating models: build, federate, or partner

The decision table below is the core of this article. Read it as a cost-and-control comparison, not a maturity ladder — most organisations end up hybrid.

Criteria	Centralised CoE	Federated model	Partner-led (buy)
What it looks like	4–6 person core team, one mandate, all AI standards set centrally	1–2 central coordinators plus AI champions embedded in each business unit	External provider runs training, use-case discovery and first pilots; small internal sponsor team
Year-1 cost band (MYR)	RM600k–RM1.4m (salaries, tooling, training, governance)	RM250k–RM600k (coordinators, tooling, per-unit training)	RM120k–RM400k (programmes, workshops, advisory)
Time to first governed pilot	14–20 weeks	10–14 weeks	4–8 weeks
Control over standards	High	Medium — drifts without a written standard	Medium — depends on contract and handover plan
Speed of decision-making	Slower; central gate	Faster; local judgment	Fastest in year one
Key risk	Becomes a toll gate; business units route around it	Inconsistent usage; data handling varies by team	Capability leaves with the vendor if there is no transfer plan
Best fit	Regulated groups, multi-entity structures, 2,000+ employees	Distributed sales or operations organisations	Firms under ~500 staff, or first-time adopters needing proof
Finance metric to watch	Cost per governed use case	% of business units with an active champion	% of capability retained in-house at month 12

Two notes that matter to the numbers. First, the centralised band assumes Malaysian market salaries for a small senior team and enterprise-tier tooling; it is a planning range, not a quotation, and your actual will move with seniority and vendor mix. Second, the partner-led band is not a cheaper version of the same thing — it buys speed and structure, and it should be judged on what it leaves behind.

Who should pick what: a straight verdict

Pick centralised if you have multiple legal entities, a regulated licence to protect, or more than roughly 2,000 employees. You need one standard, and you can absorb the slower decision cycle. For how a central function looks in practice at a professional services firm, the AI center of excellence practice write-up is a useful reference.

Pick federated if your business units genuinely differ — a distribution business where sales, logistics and finance have almost no overlapping workflows. Keep the centre thin, publish the standard, and make champions accountable for adoption in their own P&L.

Pick partner-led if you have no internal AI lead today and a board asking for a position by next quarter. This is the most common situation for Malaysian firms under 500 staff. Keep the engagement scoped so you are buying structured capability and a working pilot, not a dependency.

Pick hybrid if you are honest about the timeline: partner-led for the first 12 weeks to establish standards and run one pilot, then a small internal core team from month four onwards. This is what we typically see work, and it is the model the rollout below assumes.

The most common CFO mistake is approving a centralised structure before the organisation has enough AI fluency to staff it. A CoE with three unfilled senior roles costs the same as one with three filled ones, minus the output.

The 12-week rollout, with owners

A 12-week rollout works when each phase has a named owner and a deliverable.

Weeks 1–2 — Baseline and mandate. CFO-owned. Produce a one-page inventory of AI spend, licences, active users and current pilots. If your finance team cannot produce this in ten working days, that finding is itself the business case.

Weeks 3–4 — Use-case intake. COO or transformation lead. Open a single intake form. Score submissions on volume of hours affected, data sensitivity, reversibility and time-to-pilot. Expect 20–40 submissions and expect to fund four.

Weeks 5–6 — Model selection and budget line. CFO-owned. Choose from the table above, set the delegated spend threshold, and ring-fence the training budget separately from tooling. This is the point to confirm whether your provider can structure programmes as HRD Corp claimable — which is subject to eligibility, grant approval and HRD Corp submission requirements, and which requires your L&D team to hold the registration and claim paperwork.

Weeks 7–9 — Governance policy. Risk, compliance or legal, with a named executive sponsor. Keep it to five pages: which data can enter which tool, who reviews output before it leaves the department, what requires human sign-off, and how a possible PDPA-relevant incident is escalated. If you are working through what this looks like in practice, AIHQ runs a responsible AI and governance workshop aimed at exactly this stage.

Weeks 10–11 — Capability building. HR and L&D. Train by role, not by seniority. A finance analyst needs different habits from a customer service lead. Generic sessions produce curiosity; role-based sessions produce usage, which is the pattern behind our approach to role-based AI training and the reason it outperforms one-size workshops.

Week 12 — First governed pilot and measurement baseline. Business unit owner reports to the CFO. Report four numbers only: hours affected, cost per use case, adoption rate among targeted staff, and exception rate. No narrative slide.

For organisations that need to generate the raw pipeline before week 3, an AI innovation bootcamp is a practical way to surface use cases from the people doing the work rather than from a management offsite.

Where CoEs quietly fail

• No measurement owner. If nobody reports adoption and exception rates monthly, the CoE drifts into an evangelism function within two quarters.
• Funding pilots from a fixed annual budget. Pilots slip. Give them a small rolling allocation and a hard review date.
• Governance written before the first pilot. Policy written in the abstract tends to be either unenforceable or paralysing. Write the short version in week 7, after you know what people are actually doing.
• Training treated as an event. One workshop with no follow-up produces a spike and a fade. Role-based programmes with a 90-day follow-up produce behaviour change — a pattern worth checking against how other firms structure their generative AI practice in professional services.

What to take to your next committee meeting

Bring three things: the spend inventory, the operating model choice with its cost band, and the week-12 measure you will report. Those are the three questions your board will actually ask.

The rest is execution. AIHQ has trained and engaged over 9,000 professionals and worked across corporate, public sector, professional and regulated environments, which means the rollout above is drawn from programmes that had to survive real budget cycles. If you want to pressure-test your operating model before you commit the first ringgit, speak to AIHQ about what a structured first phase would look like for your organisation.