AI Transformation Post-Mortem: What Went Wrong

An AI transformation post-mortem should explain how a sequence of reasonable-looking decisions produced an unacceptable result. It is a structured review of what was expected, what happened, what information was available at each point and what the organisation will change. A closing presentation that merely lists lessons learned does not establish that the next programme will behave differently.

This guide is for sponsors, programme offices and assurance teams reviewing an initiative after failure, cancellation or a material shortfall. For a programme still being actively diagnosed, use the strategy diagnostic checklist. Here the focus is historical evidence, causal claims and actions whose completion can be verified.

Define the review boundary without rewriting the original promise

Record the original objective, approved scope, funding decisions and expected operating change. Identify the event that triggered the review: cancellation, a failed expansion, a serious incident or a benefit that did not materialise. State the period and systems included and which related programmes are outside the review.

Preserve the original success criteria even if they were poor. You can explain that a metric measured activity rather than value, but do not replace it silently with a more convenient goal. Separate three outcomes: whether the technology performed its defined task, whether the organisation adopted the changed process and whether the business result followed. A programme can succeed at one and fail at another.

Build the evidence timeline before interviewing for explanations

Collect the versions of the business case, scope, approvals, evaluations, delivery records and operating measures relevant to the failure. Preserve source dates and distinguish when an event occurred from when people learned about it. Interviews can explain context, but recollection should not overwrite contemporaneous evidence without a recorded reason.

Illustrative timeline for a cancelled tender-analysis programme

Point in the programme	Recorded event	Question for the review
Initial approval	Funding promises faster qualification of incoming tenders	Was the baseline and target population defined?
Pilot preparation	The sample contains recent, neatly formatted documents	Who decided which real cases were omitted?
Pilot review	Management accepts a reduction in extraction time	Did that measure establish faster qualification?
Expansion request	Three teams with different document sources are added	Was the evidence boundary reassessed?
Operating use	Review queues grow and staff maintain a parallel process	What evidence reached the sponsor and when?
Closure	The programme is cancelled after repeated scope disputes	Which earlier decision could realistically have changed the trajectory?

This is a constructed scenario, not an account of a customer engagement. Its value is the connection between events and questions. Do not begin with a predetermined explanation such as the model was poor or employees resisted change. Those statements may describe symptoms while hiding the decisions that created them.

Distinguish the trigger from the contributing conditions

A failed output may trigger the review. The deeper conditions could include an unrepresentative test set, a benefit model that ignored review work, an expansion approved without new evidence or an operating team with no capacity to accept the process. Several conditions can combine; finding one convenient cause does not establish that the analysis is complete.

Google's SRE post-mortem guidance emphasises documented contributing causes and preventive action without blaming individuals for the information available to them. This guide adapts that learning principle to programme and investment decisions. It does not imply that an organisational review replaces any separate investigation required for misconduct, contractual disputes or legal obligations.

For each proposed cause, write the mechanism. Instead of data quality was poor, explain which missing or inconsistent records affected which task, how the issue escaped the evidence review and why the team continued despite it. A cause should help identify a change that could affect the outcome; a broad label rarely does.

Test the counterfactual carefully

Ask what specific decision or control, using information available at the time, might have altered the result. A claim that leadership should have known everything the review now knows is hindsight, not a useful causal explanation. Identify the earliest point at which the relevant evidence could reasonably have been obtained and who had authority to act on it.

In the tender example, re-running the pilot evaluation on a representative set of older and Arabic documents might reveal a weakness in the original evidence boundary. That supports the claim that a different test could have exposed the issue. It does not prove that management would have changed its decision. Record that remaining uncertainty rather than presenting a reconstructed story as certainty.

Consider alternative explanations. If qualification remained slow because commercial sign-off was unchanged, better extraction alone might never have delivered the promised outcome. If a new process was usable but unfunded in operations, the problem may lie in handover rather than model performance. The stall mechanisms guide can help frame competing hypotheses.

Account for impact without inflating the loss

Separate cash spent, commitments cancelled, staff capacity consumed, service consequences and benefits not achieved. Do not count the full approved budget as a loss if much of it was never spent. Do not add all forecast benefits as a certain loss when those forecasts were themselves unsupported. Finance should review the basis and classification of the figures.

Identify reusable assets honestly. A corrected dataset, a useful integration or a trained team may retain value, but only if there is a plausible owner and use. Labelling every deliverable reusable can become another way of defending sunk costs. Record what will be retained, maintained, transferred or retired, including the responsibility for closing access and ongoing commitments.

Write actions that can be closed with evidence

Convert a learning statement into an accountable change

Weak action	Stronger proposed action	Closure evidence
Improve testing	Require the next expansion review to identify excluded case groups and test the material ones	Reviewed evaluation pack with exclusions and results
Increase ownership	Name the receiving process owner before the next pilot funding decision	Accepted responsibility and allocated operating capacity
Communicate better	Escalate unresolved scope and authority disputes through an agreed decision route	A recorded decision on a representative disputed case
Track value	Connect the next initiative's measure to the promised business outcome	Baseline and reviewed measurement plan

Give each action an owner, due date, priority, resource requirement and verifier. The review sponsor should resolve actions that nobody has capacity to implement. A long list of unfunded improvements is not a completed recovery. The steering committee operating model provides a place to track cross-programme decisions and overdue items.

Decide what restarts and what ends

A post-mortem does not automatically justify restarting the same programme with a new supplier. Revisit the underlying value proposition and the conditions for a different outcome. A restart should have a revised decision boundary, evidence requirements and ownership. If those cannot be established, closure may remain the appropriate conclusion.

Use the production transition guide for any future handover, and the assurance guide where an independent challenge is needed. Keep the learning record accessible to relevant future teams while protecting confidential and personal information. Share the conditions and decisions that can be improved, not unnecessary details about individuals.

A concise post-mortem record

The final record should contain the trigger and scope, original promise, evidence timeline, impact, supported causes and alternatives, decisions about retained assets, and actions with closure evidence. Add a review date to test whether the actions changed the next relevant programme. Without that follow-up, the organisation has documented the past without establishing a different future process.

Agentic AI consulting can help structure the review and challenge the evidence. Discuss an AI transformation post-mortem with the original decision pack, current closure position and redacted chronology. Agree who sponsors the learning and who has the authority to implement the resulting changes.