Why your AI pilot didn't pay for itself

There is a striking figure in PwC's most recent survey of chief executives: a clear majority report that artificial intelligence has produced neither increased revenue nor decreased costs in their organisation over the past twelve months. Not modest gains. Neither.

The instinctive reading is that the technology is overhyped. We think that reading is wrong, and the alternative explanation is both less comfortable and more actionable: most of these organisations bought capability and changed nothing about how they are found, chosen or bought from.

Capability without distribution does not produce a return. It never has, for any technology.

What the typical pilot actually consisted of

The pattern is consistent enough to describe generically, and most readers will recognise their own organisation in it.

A budget was allocated. Licences were bought. A cross-functional working group was formed. Some internal tooling was deployed — a drafting assistant, a summarisation workflow, a support triage layer. A handful of teams used it enthusiastically, most used it occasionally, and a report was produced at the end citing hours saved.

Now ask the question that report almost never asks: what did any of that change about whether a potential customer encounters this company at the moment they are deciding?

Nothing. The pilot made the company slightly more efficient at doing what it was already doing. It did not make the company more likely to be chosen.

Hours saved is not money

The most common measure of pilot success — time saved by staff — is the one least likely to reach the accounts.

Saved time converts into money in exactly two ways. Either the headcount goes down, or the freed capacity is redeployed into something that produces revenue. If neither happens, the time is absorbed. It becomes slightly less pressure, slightly more slack, slightly earlier finishes. Those are real goods and we are not dismissing them, but they do not appear in a profit and loss statement, and a chief executive asked whether AI increased revenue or decreased cost will correctly answer no.

Most organisations were never going to reduce headcount off the back of a pilot, and most had no plan for redeployment. The return was structurally impossible from the outset. The model was not the problem.

The distribution question nobody asked

While internal pilots were running, something material was happening outside the building. A meaningful share of the research that precedes a purchase moved into conversational interfaces. People stopped opening ten tabs and started describing their situation to an assistant and taking the shortlist it produced.

That is a change in distribution. It is the kind of change that decides which companies grow, in the same way that the shift to search decided it, and the shift to mobile decided it, and the shift to social decided it before that.

And the overwhelming majority of AI budget went to the other thing. It went to internal efficiency, in the same period that the channel through which customers discover suppliers was quietly being rebuilt.

The uncomfortable version: many companies spent their AI budget getting marginally better at producing work, while becoming invisible in the place where the work gets awarded.

Three failures, in order of cost

When we look at pilots that produced nothing, the causes sort into three groups.

The first is a measurement failure. The pilot was judged on adoption and satisfaction — how many people used it, whether they liked it — because those are easy to collect. Neither is a business outcome. A pilot with ninety per cent adoption and no revenue effect is a successful deployment of an unsuccessful investment.

The second is a scope failure. The pilot was aimed at the part of the business easiest to instrument rather than the part with the most value at stake. Internal operations are easy to pilot in. Demand generation is not. So the money went where the friction was lowest, not where the return was highest.

The third, and the most expensive, is an orientation failure. The organisation asked what AI could do for its operations and never asked what AI was doing to its market. The first question produces a tooling project. The second produces a strategy.

What a pilot with a plausible return looks like

If we were designing the next one, it would be structured around a demand question rather than an efficiency question.

• Start with a baseline of how the company currently appears when a buyer asks an assistant about its category. Not brand awareness — actual citation behaviour, sampled repeatedly, across assistants, over time.
• Compare that baseline against named competitors. Absence relative to a competitor is a concrete, defensible finding that a board can act on. An abstract statement about AI readiness is not.
• Identify the constraint. In most cases it is retrieval — the site cannot be found or read cleanly — rather than anything about content quality or brand strength.
• Fix the constraint and re-measure on a schedule. Set the success criterion before starting, in terms of citation frequency on a defined question set, and accept the answer either way.
• Only then consider paid placement, and only where the organic picture shows the conversation is happening and you are not in it.

Note what is absent from that list: no model selection, no licences, no platform decision. The work is almost entirely unglamorous, and it is aimed at the thing that determines whether revenue arrives.

The harder conversation

The reason this reframing is difficult is not technical. It is that an internal efficiency pilot is a safe project with a friendly report, and a distribution audit is a project that may conclude your competitors are being recommended and you are not.

A finding you would rather not have is worth considerably more than a report that tells you the deployment went well.

The companies we see getting somewhere are not the ones that spent the most. They are the ones that asked the demand question first, got an unwelcome answer, and fixed the foundations rather than buying another tool to sit on top of them.

If your pilot did not pay for itself, the odds are that it was never pointed at anything that could have paid. That is a fixable mistake, but only if you stop treating it as a technology problem.

The PwC figure cited at the top of this piece is drawn from published third-party research, as are the other statistics referenced across this site.