Four questions before you fund an AI pilot

The AI conversation in a mid-market company usually arrives fully formed. Somebody has seen a demo, a board member has asked what we are doing about it, and the question that lands is not should we but which vendor.

The uncomfortable part is that the technology is rarely what decides the outcome. Nearly every pilot we have been asked to look at was technically fine and organisationally stuck, and the reason was visible before anybody wrote code.

Four questions, in this order. An idea has to clear all four.

1 · Is the data reachable by a system?

Not do we have it. Every company has it. The question is whether a program can reach it on a schedule, without a person exporting anything.

“It is in the ERP” is a maybe: the ERP may or may not expose it. “Rekha pulls it every Monday” is a no, because you are not automating a decision, you are automating Rekha’s Monday and adding a dependency on her being there. And if the same figure exists in three systems that disagree, no model resolves that. Somebody has to decide which one is right, and that is a management decision, not a technical one.

2 · Is the decision worth improving?

Volume times the value of being right. A decision made four hundred times a day, where being wrong costs a rework cycle, is worth work. A decision made twice a year by a director who is good at it is not, however visible it is.

This gate is the easy one, and it is the only one most business cases actually argue.

3 · Can somebody tell right from wrong, cheaply?

Here is where the ideas start dying, and it is the gate almost nobody applies at the proposal stage.

If a model reads an invoice and pulls out a total, checking it takes two seconds and the ground truth is right there on the page. If a model summarises a contract and tells you the renewal terms are favourable, verifying that means reading the contract — which is the work you were trying not to do. The output is not wrong. It is unfalsifiable at a glance, which for anything load-bearing is worse.

The test is arithmetic: if confirming the answer costs a meaningful fraction of producing the answer by hand, the saving is smaller than the business case, sometimes by a lot.

4 · What happens when it is wrong?

It will be wrong. Not often, and not on the ten documents in the demo — on the four hundredth, with a bad scan, three months after the person who checked the first ten has changed jobs.

So: who absorbs it? If a colleague catches it in a queue they were already working through, fine. If it lands on a customer, or in a filing, or on a number that goes to the board, the honest answer to the whole idea is no — or yes with a review step, costed in, which usually changes the business case enough to reorder the priorities.

Ask it as a specific question, not a risk-register question: show me the path a wrong answer takes through this process, and name the person it stops at. If nobody can, there is no such person.

The three that usually do clear

Screened this way, the same shortlist tends to survive in mid-market estates:

• Extraction over a real backlog. Invoices, delivery notes, claims, KYC packs. The data is reachable because it is the document, the output is checkable in seconds, and errors land in a queue somebody is already reviewing.
• Search across internal knowledge. Not a chatbot — retrieval over things people already hunt for and fail to find. Wrong answers cost a search that was failing anyway, so the downside is bounded.
• Classification and routing of inbound volume. Email, tickets, applications. Misrouting is visible and recoverable, and the baseline it has to beat is a rule set nobody has maintained since 2023.

And the ones that usually do not: a chatbot on an eleven-page website, anything whose output nobody downstream is permitted to trust, and anything where the honest baseline is a database query somebody could write this afternoon. That last one is more common than it sounds. A good part of what gets scoped as AI is a reporting problem in a hat.

Two things that are not on the list

Model choice. It matters, and it is a decision you can defer and revisit — which is the opposite of the data plumbing underneath, where the choices are expensive to reverse. Anyone leading with a model is answering a question you have not reached.

Inference cost. For the workloads above it is almost never the binding constraint at mid-market volumes. Review capacity is the binding constraint, and it is the line most business cases leave out.

One thing that will be on the list by May 2027

If personal data goes into a prompt, it has been processed and, in most architectures, transferred — to a provider, in a region, under terms somebody has to be able to describe. The DPDP obligations land on 13 May 2027, and a pilot started now on customer records is a pilot that will need that answer while it is still running.

It is not a reason to avoid the work. It is a reason to know which system the data sits in before it goes anywhere, which is question one again.

What good looks like from the outside

A proposal that has been through this screen reads differently. It names two or three use cases rather than a platform, it says what it will not attempt, it puts a number on the review effort rather than assuming it away, and it identifies the reporting fix that is probably worth more than the model.

Two or three is also about what a company of this size can actually land in a year, which is a constraint worth respecting on the way in rather than discovering on the way through.