Most agencies do not have an AI idea shortage. They have several dozen candidate use cases arriving from vendors, from enthusiastic branches, and from a genuine desire not to be left behind. What they lack is a defensible way to sort them, a consistent gate structure to move them through, and a plan for the capability that has to exist underneath all of them.
This paper sets out the framework we use with clients. It is deliberately unexciting. Its purpose is to make AI investment decisions look like every other technology investment decision your organisation already knows how to make.
Why the usual approaches struggle
Two failure modes dominate.
The first is the innovation theatre approach: an innovation lab, a hackathon, a series of proofs of concept, and a portfolio of demonstrations that never enter service because nothing in the structure was designed to move work into production. Activity is high, delivered value is near zero, and the organisation becomes cynical.
The second is the enterprise platform approach, which means a large procurement for an AI platform, then eighteen months of implementation, and then the discovery that the use cases it was bought for were never the important ones. Money is committed before understanding is.
The framework below tries to avoid both by making the unit of decision a single use case, keeping the gates cheap, and being explicit that the shared capability is built incrementally out of real delivery rather than purchased up front.
Stage one: sort the portfolio
Before anything is built, get every candidate use case onto one page and score it consistently. We use four dimensions.
Value. Not "efficiency" in the abstract. What specifically changes, for whom, and how would you measure it? If the answer is a proportion of somebody's time, whose, and what will they do with it instead? Value claims that cannot survive this question tend not to survive delivery either.
Feasibility. Is this a task current capability actually does well? Retrieval with citation, classification, extraction, and drafting-for-human-review are well within reach. Multi-step autonomous action across systems of record is materially harder. Judgement with legal effect is out of scope.
Data readiness. Where does the data live, who owns it, what condition is it in, and are you allowed to use it for this? This dimension kills more use cases than any other, and it is cheapest to check first.
Risk exposure. Who is affected by an error, how would you know an error had occurred, and what is the consequence? A wrong internal search result and a wrong output affecting an individual's entitlement are not comparable risks and should not be governed identically.
The output of stage one is not a ranked list of projects. It is a segmented portfolio: a small number to proceed with, a larger number to defer with a stated reason, and a set to decline outright, including the ones better solved by a rule change, a form redesign, or a feature in software you already own.
The most valuable output of a portfolio assessment is usually the list of things you have decided not to do, and why.
Stage two: prove the hard thing, not the easy thing
For each use case that proceeds, identify the single assumption most likely to be wrong and design a short piece of work to test it. This is the opposite of the conventional proof of concept, which typically demonstrates the part everybody already believed.
In practice the riskiest assumption is rarely "can a model do this". It is usually one of:
- the data is accessible, complete and lawful to use for this purpose;
- the output can be inserted into the existing workflow without creating more work than it removes;
- the quality is high enough at a cost per item that makes sense at real volume;
- the obligations that attach to this process can be satisfied by any design at all.
Test that. Time-box it. Build inside the real environment. And define, before you start, what result would cause you to stop. A gate with no failure condition is not a gate.
Stage three: build for the second use case
The distinction between agencies that accumulate AI capability and those that repeatedly restart is whether the first production delivery leaves reusable foundations behind.
The components that recur are predictable, and they include a gateway for model access with logging and cost attribution, retrieval infrastructure with a consistent approach to permissions, an evaluation suite pattern, secrets management and monitoring. Build these as part of the first real delivery rather than as a platform programme beforehand, and the second use case takes a fraction of the time.
This is also where governance stops being a document. The gate structure, the risk classification, the human oversight requirements and the monitoring expectations should all be exercised on a real system before they are issued as policy. Policy written without a delivery to test it against is usually either unworkable or ignored.
- A typical stage one portfolio assessment
- 3 to 4 weeksA typical stage one portfolio assessment
- Stage two proves the idea against real data
- 4 to 6 weeksStage two proves the idea against real data
- The first delivery leaves reusable foundations
- 3 to 6 monthsThe first delivery leaves reusable foundations
Stage four: govern the portfolio, not just the projects
Once more than one AI system is in service, the governance question changes. You now need a standing view of what is running, on what data, with what oversight, and how each system is performing against the baseline it was accepted on.
This is a register, an owner for each entry, a review cadence, and a defined trigger for re-assessment when a model or a business process changes. It is not sophisticated. It is the same discipline you apply to any other set of production systems, and the absence of it is what turns a successful AI programme into an audit finding two years later.
Capability, not just systems
The framework has one further requirement that is easy to skip: at each stage, name what capability your organisation is acquiring, and who holds it.
If an external partner runs the portfolio assessment, someone internal should be able to run the next one. If a vendor builds the first system, your engineers should have been alongside them. If the evaluation suite was written for you, your team has to be able to extend it when the task changes.
Agencies that treat capability transfer as a deliverable rather than a courtesy end up with a functioning internal practice in about two years. Those that do not end up with a set of systems they cannot modify and a procurement dependency they cannot easily exit, which in the public sector is a governance problem rather than merely a commercial one.
Summary
Sort the portfolio honestly and be willing to decline things. Test whichever assumption is most likely to fail, even when a different one would make a better demonstration. Leave reusable foundations behind on the first delivery. Govern the portfolio as a standing responsibility. And be explicit about who ends up holding the capability.
None of this is novel. That is rather the point: AI adoption goes wrong most often when organisations treat it as exceptional and abandon the disciplines they already apply competently to everything else.
- Strategy
- Governance
- Government