Many artificial intelligence projects are approved on the basis of strategic intuition, a passing fad or a benefit deemed «obvious», but rarely rigorously measured. For a CEO, a CFO or a digital transformation lead, the real question is not just whether AI creates value, but how to prove it. Discover the methods, decision-making protocols and pitfalls to avoid when moving from gut feeling to hard evidence, with Gauthier Lamothe, entrepreneur and trainer in generative AI.
According to McKinsey, Fewer than 20 % of organisations currently track indicators specifically relating to generative AI. This is a situation that should give us pause for thought, as the budgets allocated to these projects are skyrocketing. This discrepancy between the scale of investment and the lack of effective management explains why so many companies struggle to answer a simple question: how much does AI actually generate?
Here are some practical guidelines for:
- distinguish between perceived benefits and measurable gains
- Choosing the right AI strategy for your use case
- to establish a reliable measurement framework prior to any roll-out
Note: In this article, we will focus solely on generative AI, as this sector has really taken off over the last decade. The return on investment for traditional machine learning algorithms is often better documented in certain use cases (scoring, fraud detection, recommendations, forecasting, optimisation, etc.), but it is never automatic. It depends on data quality, integration costs, maintenance, model drift and business adoption.
Why the ROI of generative AI cannot be measured using traditional methods
There are four biases that explain why it is difficult to assess an AI project accurately:
- The lack of before-and-after measurements prevents any rigorous comparison.
- Over-reporting or under-reporting of earnings distorts the data collected in the field.
- Some key performance indicators (KPIs) are simply not included in the ROI calculation.
- Finally, the confusion between an isolated trial and a full-scale roll-out skews the projections.
These biases are not merely anecdotal. The report MIT " The GenAI Divide » notes that 95% corporate initiatives have had no measurable impact on the profit and loss account. Not because the models are useless, but because they are poorly integrated into actual workflows.
The investigation McKinsey State of AI 2025 The figure cited earlier points in the same direction: more than 80 % of organisations report no tangible impact on EBIT at company level, even though generative AI is now used in an average of three business functions.
The gap between perceived ROI and actual ROI
The most costly pitfall is the gap between what teams think they will earn and what they actually earn.
The study METR A study carried out in 2025 on 246 real-world tasks assigned to experienced open-source developers equipped with AI tools clearly illustrates this. The AI-assisted group took 19 % longer than the control group. However, these same developers estimated afterwards that they had saved 20 %. The gap between perception and reality stands at 39 points.
The lesson to be learnt is that without quality metrics, the reported ROI may be an illusion.
| Study | Methodology | Measured result | Discrepancy with perception | Education |
|---|---|---|---|---|
| METR (2025) | RCT, experienced developers | −19 % speed | +39 points compared with the reported estimate | The perception of productivity can be misleading. |
| BCG × Harvard / MIT / Wharton (2023) | RCT, 758 consultants | +12.2 % of tasks completed | −19 % of precision outside the scope controlled by the AI, without users realising it | AI is a great help with certain tasks but can actually make other, similar tasks worse. |
| Brynjolfsson–Li–Raymond, NBER (2025) | Phased roll-out, 5,179 staff members | +14 % resolved issues per hour | +34 % for beginners, virtually no gain for experts | Earnings vary depending on the level of experience. |
This final point is worth bearing in mind for any finance director budgeting for an AI project: winnings are not consistent.
They often benefit less experienced staff the most and may be low, non-existent or negative for experts, depending on the task.
«With our AI pilot scheme for drafting tenders, junior account managers saved nearly half a day per project: the AI helped them structure the response, find internal references and produce a usable first draft.”.
»Among the older participants, the effect was much weaker. They were already familiar with the references, knew how to formulate the arguments and, above all, spent their time correcting inaccuracies."
Marc, Finance Director of a B2B services group
Three pillars for structuring ROI measurement
Leading consultancies are adopting a common framework, despite using different methodologies. They all measure the ROI of generative AI based on three key pillars.
| Pillar | Measured phenomenon | Example of an indicator |
|---|---|---|
| Productivity | Speed of execution, volume processed, time saved | Tasks per hour, PRs per week, tickets resolved per hour, time per case |
| Quality | Reliability, accuracy and consistency of deliverables | Error rate, escalation rate, hallucination rate, human intervention rate |
| Business value creation | Direct or indirect business impact | Increase in revenue or margin, cost per interaction, NPV, payback period |
Assessing a project on the basis of a single criterion gives an incomplete picture. A chatbot may demonstrate excellent productivity whilst undermining the quality perceived by the customer.
The Klarna case is instructive in this regard: the company had announced a gain equivalent to 700 full-time jobs, before its CEO acknowledged a reduced service quality and did not resume recruiting staff until fifteen months later.
This is a clear illustration of Goodheart’s law in project management: naively optimising a KPI by disconnecting it from the phenomenon it purports to measure inevitably leads to an antitelia (an unintended consequence where a decision produces the opposite of what was expected).
As a consultant, five years ago I only came across such problems occasionally, whereas now, in 2026, I see about one a week. This shows that the issue has not received as much attention as the implementation of generative AI in business processes.
Build, Buy or Boost: choosing the right AI strategy for the use case
Not all tasks are equal when it comes to generative AI. The concept of «jagged technological frontier» (jagged technological frontier), developed by the BCG, describes a simple reality: the models perform exceptionally well on certain tasks but fail on similar ones, with no apparent logic for the user.
This irregularity means that a well-considered decision is required before any investment is made. You have three options available to you.
| Strategy | When to use it | What this means |
|---|---|---|
| Build (in-house AI) | The task is beyond the reliable capabilities of generic AI models, or represents a strategic advantage | Proprietary data, dedicated architecture, fine-tuning, in-house R&D |
| Buy (AI subscription) | The task is well covered by standard models, with no competitive advantage to be gained from in-house production | Use of an existing API or SaaS solution, limited integration, predictable costs |
| Boost (contextual AI) | The task is well covered, but could be improved considerably with your own data | RAG, business connectors, document repositories, data governance |
This protocol avoids the most common pitfall: developing a solution in-house that is already available on the market as a SaaS offering, or, conversely, outsourcing a task that required distinctive expertise to a generic AI system.
Another safety measure worth mentioning is the one adopted in France by the AP-HP: rejecting business cases that merely aim to «save 5 to 10 minutes». Seeking such significant gains (tasks that are currently impossible or highly complex) helps to ensure that the implementation of processes involving generative AI will not be carried out at a loss.
The true cost of an AI deployment: TCO, hallucinations and maintenance
ROI is never calculated on the basis of profit alone. Gartner breaks down the total cost of ownership (TCO) of a generative AI project into four layers:
- Licences, APIs and subscriptions
- infrastructure,
- human capital,
- hidden costs of compliance and governance.
According to the survey KPMG AI Quarterly Pulse, average spending on GenAI by large companies has almost doubled in a year – a rate of investment that is rarely set against the actual gains measured.
The hidden cost: checking, correcting, monitoring
Hallucinations are one of the best-documented hidden costs. The study Stanford RegLab/HAI measure of the hallucination rates of 17 to 33 % in specialist legal tools, and from 69 to 88 % on mainstream models used for specific legal matters.
This figure is worth highlighting, as just 18 months ago, “legal” use cases were seen within the generative AI industry as the prime area where productivity gains were expected to be realised.
When a mistake becomes a financial risk
These errors come at a direct cost: the law firm Morgan & Morgan was fined $5,000 for eight out of nine fictitious citations in a legal brief generated by AI.
Another textbook example that I think it is worth mentioning is that of Deloitte. In October 2025, the firm was required to make a partial refund for a report worth 440,000 Australian dollars commissioned by the federal government, following the discovery of non-existent academic references and a fabricated citation attributed to a court ruling. A month later, a report commissioned by a Canadian provincial government faced similar allegations involving a sum of 1.2 million Canadian dollars.
These two incidents show that the cost of a mistake is not limited to correcting a text; it can also lead to legal, reputational and financial risks.
The Google precedent: AI binds the party that publishes it
At the end of the first half of 2026, Google itself had to deal with the consequences of its AI’s delusions: the Munich Regional Court ruled that the Google’s AI Overviews are Google’s own statements, rather than simply reproducing third-party content.
The result is that Google may be directly liable false or defamatory information generated by this feature.
The court explicitly distinguished between traditional search engines (which link to third-party pages) and AI Overviews (which summarise and generate new text).
The case arose because two German publishers were identified by the AI as being linked to scams and fraudulent practices, even though these claims did not appear in the sources consulted. The court’s reasoning is as follows:
- A traditional search engine acts primarily as an intermediary; ;
- An AI Overview (AI summary) writes a new text, decides which information to retain and formulates original statements; ;
- These statements are therefore regarded as Google’s own content.
Google had defended itself by arguing that users are aware that AI can make mistakes (a warning is explicitly provided), but the court rejected this argument, stating that a warning is not sufficient to exonerate Google when the system itself produces a false statement that damages the reputation of a person or a company.
The difference lies in governance
At the other end of the spectrum, a case such as JPMorgan COIN demonstrates that rigorous implementation can reverse the trend: 360,000 lawyer-hours saved each year across 12,000 commercial contracts, with a reduction of approximately 80 % in loan management errors. The difference between these two trajectories lies not so much in the technology itself as in the governance framework surrounding it.
Establish a measurement framework before any deployment
A reliable ROI is planned before the project is launched, not after. Here is the process to follow:
- Identify the tasks that are actually enhanced by AI, distinguishing those with high potential from those outside the reliable technological frontier.
- Define 3 to 5 measurable indicators prior to roll-out, aligned with the relevant business value unit.
- Set up a before-and-after comparison, ideally with a control group, using a real-world task rather than a simulated one.
- Include the full cost : licences, infrastructure, training, governance, incident monitoring and resolution.
- Setting a threshold for scaling up, for example, a proven benefit over a 90-day period prior to any wider roll-out.
- Schedule an independent audit at 6 and 12 months, to detect a decline in quality before it becomes a problem.
Summary table of indicators to monitor
| Category | Examples of indicators |
|---|---|
| Usage | Adoption rate, frequency of use, acceptance rate of suggestions |
| Productivity | Time per task, throughput (cases per hour, tickets per hour) |
| Quality | Error rate, escalation rate, hallucination rate, human intervention rate |
| Experience | Customer satisfaction (CSAT), NPS, customer churn |
| Finance | Cost per interaction, net return on investment, payback period |
| Systemic | Talent retention, teams’ ability to adapt quickly |
[Also read] Best practices for working in a team with AI
The ROI of GenAI is measurable, but varies greatly. Controlled studies show actual gains of between 14 and 56 % on specific tasks. However, the majority of corporate pilot projects currently have no measurable impact on the profit and loss account.
Measuring the ROI of an AI deployment therefore requires moving beyond self-reported figures to establish a rigorous framework even before the first pilot. Distinguishing perceived benefits from proven gains, choosing the right strategy for each use case, and factoring in the full cost – including ‘hallucinations’ – is what separates a profitable AI project from one that merely gives the impression of being so.
