Why AI Readiness Assessments Fail

An AI readiness assessment fails when its conclusion cannot be supported by the evidence it gathered, does not answer the decision the customer needed to make or leaves the organisation unable to act on the findings. A conclusion that a proposed workflow should pause can be a successful assessment. A positive score can be an unsuccessful one if the team cannot explain what was examined, what remains unknown or why the next step is justified.

This guide helps Saudi sponsors and procurement owners diagnose the assessment itself before commissioning more AI work. Its illustrative case is a food wholesaler considering harmonisation of supplier price lists for internal comparison. The case concerns an assessment that treated visible spreadsheet values as sufficient evidence for comparable prices. It is a fictional diagnostic example, not an allegation about a named assessor or a claim about failure rates in Saudi businesses.

Start with the decision the assessment was supposed to support

The wholesaler wanted to know whether a controlled trial could compare selected supplier offers on a consistent basis. The intended output was an internal comparison for a buyer to review, with unclear units returned for clarification. It was not an automated purchasing system, a pricing engine or authority to accept a supplier's commercial terms. This boundary determines which evidence the assessment needed.

Imagine the final report instead says the company is “AI ready” because it uses spreadsheets, has management support and can access a modern model. None of those observations establishes that the selected prices are comparable. The first diagnostic question is therefore whether the report answers the agreed decision or substitutes a broader, easier question about general enthusiasm and technology availability.

Recover the original brief, proposal and meeting decisions. Identify changes to scope and whether they were accepted. A report may be unsuitable because the decision changed during the engagement, because the assessor misunderstood it or because the customer never defined it. The remedy depends on that history. Avoid assigning blame before establishing which question was actually commissioned.

Trace one material conclusion back to its evidence

Choose a consequential statement, such as “supplier prices can be compared after standard formatting”. Ask which files, rows, definitions and owner decisions support it. The trace should reach observed material, not stop at another summary slide. Record whether the statement is an observation, an inference or an assumption awaiting confirmation. These categories help locate where confidence was introduced without evidence.

In the illustrative case, every price cell in twenty-four selected rows contains a number. Nine rows lack a verified pack quantity, even though their descriptions look familiar. A completeness statement about populated price cells can be correct while the conclusion about comparable unit prices is unsupported. The nine unresolved rows are a finding about this diagnostic selection, not an estimated prevalence across all supplier records.

The NIST AI Risk Management Framework distinguishes difficulty measuring risk from evidence that risk is high or low. The same caution is useful here: absent evidence should remain visible as uncertainty, rather than becoming a favourable assessment result by default.

Reconstruct the point where meaning was lost

A readiness failure may occur between correct observations rather than inside an obviously false statement. The assessor sees a populated spreadsheet, the business owner confirms that buyers use it and a demonstration extracts the numbers. Each observation may be true. The unsupported step is the inference that extracted numbers have a common commercial meaning and are suitable for comparison.

Illustrative evidence chain for a price-list readiness conclusion

Recorded item	What it supports	What it does not establish
Price cell contains a number	A value is present	The unit, currency or applicable conditions are comparable
Buyer recognises the supplier	Operational familiarity	Authority of this particular file and revision
Extraction demonstration succeeds	The demonstrated values can be read	Correct business interpretation across unexamined lists
Manager supports the project	Sponsorship exists	A named owner can resolve pack and validity conflicts
Trial scope is agreed	A proposed boundary exists	All conditions for attempting it have been evidenced

This reconstruction is more useful than arguing about whether the assessment should have asked more questions. It identifies the exact inference that needs repair. The follow-up can then request the missing unit definition or source decision rather than repeat a general workshop that produces another optimistic summary.

Check whether the sample was convenient rather than informative

The wholesaler has twelve illustrative supplier lists across three branches. Suppose the assessment examined two recently cleaned lists supplied by the project champion and treated them as representative of the whole task. The selected material may be appropriate for a narrow first demonstration, but it cannot establish conditions in the remaining lists without a defensible explanation of coverage.

Ask which differences were deliberately included: packaging conventions, revision histories, language, spreadsheet structure and unresolved supplier notes. Also ask which were excluded and why. The objective is not to inspect every file regardless of cost. It is to ensure that the evidence supports the actual conclusion and that unexamined conditions remain clearly outside it.

Use sample quality and representativeness to plan a focused extension when coverage is inadequate. Preserve the original selection rationale so the team can distinguish an agreed limitation from an unnoticed blind spot. A small, transparently bounded assessment can be useful; a small selection presented as universal evidence creates a different problem.

Look for a score that has replaced an unresolved condition

A combined score can create the appearance of precision while hiding the decision that matters. Strong sponsorship and available technology do not compensate for an unknown pack quantity if the proposed task compares unit prices. The diagnostic question is whether the report treated all categories as interchangeable contributions to one total, even when a condition was essential to the workflow.

For example, an illustrative offer of one hundred riyals per carton becomes ten riyals per item only if the carton is confirmed to contain ten relevant items and the comparison conditions align. Without that evidence, dividing by an assumed quantity produces a precise but unsupported value. A high overall readiness score cannot supply the missing commercial definition.

Use gates versus scores to restore the distinction between prerequisites and useful improvements. A finding can block a specific comparison path without condemning every part of the organisation. The repaired conclusion might permit extraction and source inspection while withholding comparative ranking until units and applicable conditions are confirmed.

Identify conclusions that quietly assume future remediation

Some reports describe the workflow as ready because missing elements could be fixed. A buyer could confirm pack sizes, IT could create access and a manager could allocate review time. Those may be reasonable remediation proposals, but they are not evidence that the conditions currently exist. Check whether the conclusion has silently converted future work into present capability.

Rewrite each such statement with its status. A source owner proposed is different from a source owner appointed and available. A possible conversion rule is different from an approved rule tested on the relevant cases. A trial environment requested is different from one that the authorised participants can actually use. The status should affect what the next decision permits.

Do not require every improvement to be complete before the assessment can deliver value. A conditional recommendation is legitimate when it names the remaining work, owner and acceptance evidence. The failure is claiming that the condition is already met or leaving it so vague that no one can close it. A remediation backlog can turn that ambiguity into an actionable plan.

Inspect whether the demonstration answered the wrong question

A demonstration can be technically successful and irrelevant to readiness. Reading supplier prices from a clean file tests extraction on that file. It does not establish whether conflicting supplier notes can be resolved, whether the current file is authoritative or whether the buyer can review the result at the planned volume. Preserve the demonstration result, but narrow the claim it supports.

Ask who selected the examples and defined success before the demonstration. If expected behaviour was decided after observing the output, the team may have accepted whatever the tool happened to produce. Include cases where the proper result is an unresolved comparison or a request for clarification. Otherwise the demonstration rewards completion even when completion requires an unsupported assumption.

Use readiness versus proof of concept to distinguish the prerequisites for a useful experiment from the experiment's findings. An assessment does not need a working prototype to identify a missing source decision. Equally, a prototype that looks promising does not prove that all business, access and operating prerequisites have been established.

Check whether the people interviewed could settle the issue

An interview records what someone says, not necessarily an authoritative decision. The project champion may understand the spreadsheets without owning supplier terms. A branch buyer may know local conventions that do not apply elsewhere. Ask which conclusions depend on a person's knowledge and whether that person can confirm the applicable rule for the proposed scope.

In this case, one branch routinely interprets a carton code from experience. Another uses the same code for a different pack arrangement. An assessor who records only that “buyers understand the codes” misses the need to define which convention governs each record. The missing evidence may be a documented supplier clarification or a source-owner decision, not another opinion survey.

Keep disagreement visible. Do not resolve it by using the most senior interviewee's statement unless that person has the relevant authority and evidence. The assessment should show the unresolved issue and its effect on the next step. Ownership and evaluation readiness helps identify the decisions that require named, operationally available owners.

Separate an assessment defect from an agreed limitation

A report that clearly excludes some branches is not necessarily defective because those branches later prove difficult. A report that claims to cover them without examining relevant evidence may be. Compare the delivered work with the agreed scope, methods, assumptions and acceptance criteria. The distinction matters when deciding whether to request a correction, commission an extension or reconsider the original brief.

Also distinguish new facts from overlooked facts. If a supplier changes its pack structure after the assessment, a focused update may be appropriate. If the original files already contained conflicting pack information that the assessor ignored, the conclusion may need correction. Preserve dates and source revisions so the review can establish what was knowable when the decision was made.

Use a neutral finding format: statement reviewed, evidence available at the time, unsupported inference or excluded condition, decision affected and proposed remedy. This keeps the discussion specific. General accusations about quality rarely help the customer obtain a better decision, while a traceable correction request can identify exactly what must change.

Repair the conclusion before repeating the whole engagement

The first response to a weak assessment should not automatically be another full assessment. Determine whether the gap is local and recoverable. The price-list case may need targeted verification of pack definitions, source validity and reviewer responsibility. If the underlying evidence is available and the scope is clear, those tasks may support a revised, narrower conclusion.

Illustrative repair plan for the wholesaler's assessment

Defect	Focused repair	Evidence of completion
Unknown pack quantities treated as comparable	Verify the affected records or exclude them	Recorded source and decision for each unresolved comparison
Two clean lists generalised to twelve	Review relevant differences or narrow the conclusion	Coverage rationale and explicit exclusions
No owner for source conflicts	Assign an authorised resolution route	Named role, available evidence and accepted responsibility
Positive score obscures prerequisites	Rewrite the next-step decision with conditions	Traceable conclusion linked to unresolved requirements

A restart may be justified when the decision itself is unclear, evidence cannot be recovered or the method systematically prevents challenge. Explain that reason rather than treating repetition as inherently safer. The customer should know which uncertainty the additional work will resolve and how acceptance will differ from the first engagement.

Agree how the repaired assessment will be accepted

Acceptance should focus on the usefulness and supportability of the decision. Ask an informed colleague to trace a material conclusion, locate its source and explain its limit. Ask an operational owner to identify the next task and the evidence that would close it. If these steps cannot be performed, a polished report may still leave the customer unable to act.

Include contradictory evidence in the review. A revised conclusion should explain how the unresolved rows affect the permitted trial, not simply append them while keeping the original positive headline. Check that summaries, tables and recommendations say the same thing. A condition buried deep in an appendix may be lost when an executive forwards only the decision page.

The deliverable can be accepted even when the proposed AI workflow should pause. The assessment's job is to support a proportionate decision, including a decision not to proceed yet. Do not make assessor payment or acceptance depend solely on a positive readiness outcome; that can undermine the incentive to report an inconvenient but material finding.

Use the failure review to choose a proportionate next investment

The corrected assessment should leave the sponsor with alternatives, not merely a longer list of problems. For the wholesaler, one option is to pause comparative ranking and test extraction with clear unresolved-unit flags. Another is to verify a smaller set of supplier offers and attempt comparison only within that set. A third is to invest first in a maintained source-definition process because the ambiguity affects several buying activities beyond AI. Each option answers a different business question.

Compare the evidence needed, the owner effort and the decision each option could support. Do not assume that the broadest repair is automatically the best use of budget. If the immediate decision concerns one category of supplies, proving every supplier convention may be unnecessary. If the same unresolved definitions repeatedly disrupt purchasing, a narrow demonstration may avoid the underlying operational issue. The sponsor needs to see that tradeoff explicitly.

Keep sunk assessment cost separate from the next decision. Money already spent does not make an unsupported conclusion more reliable, nor does it require abandoning evidence that remains valid. Preserve useful observations, correct the overreach and decide which additional work has a clear purpose. This makes the review commercially constructive rather than a debate over whether the entire engagement should be called a success or a failure.

Record the lesson at the level where it can change future work. “Ask better questions” is difficult to implement. “Do not accept a unit-comparison conclusion without a verified unit definition and source for each included case” is testable. The lesson should improve the next brief, evidence request or acceptance check, while remaining specific enough that an owner can demonstrate it has been applied.

Prevent the same failure in the next assessment brief

Define the next-step decision, evidence boundary, sample rationale and required deliverables before work begins. Ask for findings that distinguish observed conditions, assumptions and unassessed areas. Require material conclusions to be traceable and make the treatment of blocking conditions explicit. These requirements are more useful than demanding an impressive page count or a universally favourable score.

State which customer roles and materials must be available. If access or an owner is missing, agree how the assessor should record the limitation and whether it changes scope or timing. Use assessment preparation to reduce avoidable gaps without sanitising the evidence. The assessor should see difficult examples and uncertainty, not only a carefully selected presentation of the business.

Connect the revised work with Enterprise Data & AI Readiness. A readiness review enquiry can begin with the original decision, the conclusion that concerns you and one example of evidence that appears inconsistent with it. That provides a practical basis for diagnosing whether the problem is the assessment, the underlying workflow or a change since the original review.