Commission the right report
AI Audit vs AI Assurance
Separate technical audit findings, broad AI assurance programmes and formal professional assurance before commissioning a report.
“AI assurance” can mean a broad programme for building justified confidence in AI, or a formal professional engagement with a prescribed type of conclusion. “AI audit” usually describes an examination against agreed criteria, but the label alone does not identify its professional status. Buyers should establish the intended users, subject matter, criteria, evidence and report form before deciding which service they need.
The distinction matters when an internal technical report is later presented to a customer, board, lender or regulator as if it were an independent assurance opinion. The underlying work may be useful, yet the claim made about it may be unsupported. This guide explains how to commission the right deliverable and preserve its meaning when it moves between audiences. It does not present Open for AI's technical assessments as regulated assurance opinions or certification.
Two meanings of assurance that buyers must separate
The UK government's introduction to AI assurance uses the term broadly for evaluating and communicating evidence about AI trustworthiness. In that sense, testing, impact assessment, audits and other techniques can contribute to an assurance programme. This is a useful conceptual reference, not a Saudi legal requirement or a promise that any one technique is sufficient.
Formal professional assurance is more specific. The IAASB's ISAE 3000 (Revised) addresses assurance engagements outside audits or reviews of historical financial information. Its professional framework cannot be adopted merely by adding the word “assured” to a technical report. The appropriate practitioner must determine whether the subject matter, criteria, evidence, competence, ethics and other engagement conditions support the work requested.
A buyer can therefore ask two distinct questions: “What evidence increases confidence in our AI system?” and “Do we need a formal assurance conclusion that a specified recipient will accept?” The first can lead to a technical audit or a combination of evaluations. The second requires confirmation of the applicable professional requirements and provider eligibility before a proposal is accepted. Neither answer should be inferred from marketing terminology.
A practical comparison of deliverables
| Work requested | Useful output | Boundary to confirm |
|---|---|---|
| Scoped technical AI audit | Findings, tested criteria, supporting evidence, limitations and retest requirements | No automatic statutory opinion, certification or formal assurance conclusion |
| Organisation-wide AI assurance programme | Coordinated evidence from different reviews, owners and monitoring activities | The programme label does not give every component the same independence or reliability |
| Formal professional assurance engagement | A report in the agreed professional form from an eligible practitioner | Standard, level, subject matter, criteria, users and restrictions must be explicit |
| Management-system certification | Certification decision within the certification body's authorised scope | Scope and scheme differ from a technical audit of an individual AI application |
These categories can coexist. A technical specialist may supply work that a formal assurance practitioner considers, but the practitioner remains responsible for deciding whether and how that work can be used. A management team may maintain an assurance programme containing both independent and management-produced evidence. The important discipline is to preserve each component's provenance and limits rather than merge them into an unexplained confidence score.
Example: a supplier sustainability narrative generator
Consider a hypothetical procurement organisation using an AI application to draft supplier sustainability summaries from approved questionnaires and supporting documents. A human editor checks the draft before it enters an internal supplier review. The system is not authorised to verify emissions, certify supplier sustainability performance or issue an assurance opinion. Its task is to describe the supplied evidence without inventing achievements or concealing uncertainty.
The operational owner wants to know whether citations support generated statements and whether unsupported claims reach a reviewer. A technical audit can examine that workflow using selected documents, generated drafts, editor records and application configuration. A downstream customer, however, may ask for formal assurance over a published sustainability statement. That is a different subject matter and intended use, potentially requiring different standards, competence and evidence.
The fact that the generator was technically audited does not establish that the supplier's underlying claims are true. A system can faithfully reproduce an inaccurate questionnaire. Conversely, a correct underlying statement can be misrepresented by the generated narrative. Separate source reliability, transformation accuracy and external reporting assurance. Otherwise a report about the middle of the chain may be incorrectly used to endorse both ends.
Write the statement that the recipient actually needs
Before buying a service, ask the intended recipient what decision it must make and what form of report it requires. “We need assurance” is insufficient. Does the recipient want evidence of tested controls, an independent professional conclusion on specified information, a certification within a scheme, or simply management's account of remediation? Different answers change the engagement and the provider selection.
For the hypothetical generator, an internal statement might read: “The review examined whether selected generated summaries were supported by the approved source records in application version X.” That is different from: “The supplier sustainability information is reliable in all material respects.” The second statement reaches beyond the generator and would require the relevant practitioner to determine a suitable engagement. Do not ask a technical report to carry a broader meaning through a change of heading.
Document who is responsible for the underlying information and who evaluates it. Management remains responsible for the system and its use. A reviewer can examine evidence without assuming ownership of the business decision. If roles change during the work, update the engagement rather than allow an adviser who designed the information to appear later as an unqualified independent evaluator of the same work.
Criteria make the conclusion interpretable
A criterion defines what counts as meeting the claim. “Good AI governance” is too vague on its own. “Every published supplier summary identifies the source record for each quantitative claim, with unsupported claims held for review” is more specific. The criterion needs definitions of published, source record, quantitative claim and held for review, plus a period and population.
Criteria can be unsuitable even when easy to measure. Counting citations does not establish that citations support the statement. Requiring an editor name does not establish that the editor reviewed the final text. A formal assurance practitioner assesses the suitability of criteria under the relevant requirements; a technical audit should also disclose the meaning and limitations of its agreed criteria. Both need enough precision that another competent reader can understand the result.
Custom criteria may be useful for an internal decision but unsuitable for unrestricted external reliance. If they were written around a narrow configuration or negotiated tolerance, readers need access to that context. A favourable conclusion against a weak criterion can be technically accurate yet commercially misleading. Procurement should ask to see the criteria before the final report, not only a sample cover page.
A claim-to-report translation exercise
| Proposed wording | Problem | More defensible description |
|---|---|---|
| Our AI is independently assured | No subject matter, provider role, criteria or report form is identified | Name the actual review, responsible provider, tested version and scope |
| The audit proves suppliers are sustainable | The generator review did not verify underlying supplier performance | The audit examined selected source-to-summary controls, with stated exceptions |
| No issues were found, so the system is compliant | Absence of observed issues is widened into comprehensive legal compliance | Report the procedures, results and areas on which no conclusion was possible |
| The report can be used by any customer forever | Users, purpose, versions and time limits are ignored | Check permitted use and whether changes require further work |
This exercise is useful before an external announcement or sales questionnaire response. Ask the technical owner and the relevant professional adviser to review the exact sentence, including nearby headings and badges. A carefully qualified paragraph can still be misleading if a prominent headline says “certified AI”. Preserve the report's scope wherever the organisation refers to it, and do not imply provider endorsement of unrelated products.
Reasonable and limited assurance are professional terms
Where a formal assurance engagement is appropriate, reasonable and limited assurance are distinct levels under the relevant professional framework. They are not synonyms for “expensive audit” and “cheap audit”, and neither means absolute certainty. The practitioner determines the procedures and conclusion appropriate to the agreed engagement. A buyer should not select the wording first and then ask a technical team to reverse-engineer a matching opinion.
The IAASB's explanation of non-financial assurance emphasises suitable criteria, professional competence and ethical conditions alongside the distinction between levels. For an AI-related request, ask the proposed practitioner to explain how those requirements apply to the exact subject matter. Do not assume that an engagement about system controls is interchangeable with assurance over information produced using that system.
A report's level also does not remove scope exclusions. Formal assurance on specified information can coexist with an unexamined model component. A technical assessment can contain deep testing without becoming a formal assurance report. Depth, independence, professional form and scope are separate dimensions; a procurement comparison should show all four rather than rank providers on a single “assurance strength” label.
How evidence from different reviewers can be used
An assurance programme may include developer evaluations, internal audit work, supplier reports, independent technical testing and operating monitoring. Each answers a different question and may cover a different period. Establish an evidence map that records the producer, purpose, criteria, population, version, access and unresolved limitations. Then identify the claim to which the item contributes.
For the generator example, a supplier security report may support a narrow hosting-control question. It does not establish source-to-summary accuracy. An editor's review log may support evidence of human involvement, but not necessarily the quality of that review. A technical test may reveal an unsupported citation while leaving the authenticity of the original supplier document unresolved. The map should preserve these distinctions instead of treating document volume as sufficient assurance.
If another practitioner is expected to rely on technical work, involve them before testing begins. They may need different selection methods, workpaper detail, access or review rights. Repackaging a completed report afterward can leave gaps that require repeated work. Supplier evidence quality and working-paper traceability are useful when designing an evidence package for a clearly identified downstream reader.
Independence and competence cannot be borrowed from a label
Ask whether the provider built the application, designed the criteria, selected the data or will sell the remediation. Those relationships do not make every review worthless, but they can affect the independence claim and the safeguards required. A team may be competent to explain a system while not being the right party to give the independent conclusion a recipient demands.
Provider eligibility for a formal engagement must be checked against the applicable professional and jurisdictional requirements. A technical consultancy should not imply that its ordinary assessment provides a regulated opinion. If several firms participate, name who owns each deliverable and the final conclusion. Independence and conflict-of-interest review should occur before contracting, and again if the provider's role changes during remediation.
Certification is a separate procurement route
Management-system certification concerns a defined management-system scope and certification process. It should not be treated as proof that every model output is correct or that every application has been technically audited. Equally, a detailed technical audit does not automatically produce a management-system certificate. If a customer requests a named certificate, establish the relevant certification route and eligible body rather than offering a differently named technical review.
The dedicated comparison of AI auditing and ISO 42001 certification explains that boundary in more detail. A technical audit may help identify evidence gaps before another assessment, but readiness work and the certification decision remain distinct. Marketing teams should be given the exact permitted description of each outcome, including the scope to which any certificate applies.
Check report portability before reusing a conclusion
A report prepared for one reader can be unsuitable for another even when the underlying work is sound. The original reader may understand private criteria, a narrow workflow or an agreed limitation that a new customer does not know. Before sharing, ask whether the new purpose changes the claim, the evidence needed or the professional form expected. Permission to circulate a document is not the same as confirmation that the new recipient can rely on it for a different decision.
In the sustainability-generator example, an internal procurement team may use the technical report to decide whether to retain source checking before summaries are circulated. A customer may instead want confidence in a published supplier statement. The first report could be relevant background, but its existence does not answer the customer's question. The organisation should explain the distinction and identify the additional work, rather than attach the report to a broad assertion and let the recipient infer endorsement.
Review accompanying material as carefully as the report itself. Sales decks, tender responses and executive summaries can turn a bounded statement into an unrestricted claim. A phrase such as “independently reviewed” needs enough nearby context to identify what was reviewed and the relevant limitations. Do not remove a qualification because it makes a slide less persuasive. If the description cannot be understood without the full report, provide access under the agreed terms or use a narrower description.
Build readiness for the intended professional route
If a formal assurance engagement is likely to be required, ask the appropriate practitioner to identify readiness gaps early. Those may concern unclear responsibility for the information, unsuitable criteria, incomplete evidence or inability to provide access. Fixing such gaps can be valuable even before the formal engagement begins. Readiness work should state what was improved and what remains for the practitioner to assess; it should not announce that a future assurance conclusion is guaranteed.
The organisation should preserve evidence as the system operates, not attempt to reconstruct an entire history when an external report is suddenly requested. For the generator, retaining source versions and review records may be necessary to understand past outputs. A current successful demonstration cannot recreate missing historical evidence. The practitioner may need to narrow the period or subject matter, or the organisation may need to wait for an adequately evidenced operating period.
Keep the commercial sequence explicit: technical readiness work, any remediation, the formal engagement and later updates are separate decisions. This avoids paying twice for incompatible procedures and avoids presenting preparation as the final outcome. The intended reader, responsible party and relevant professionals should understand the same boundary before the organisation makes a public claim about the result.
A commissioning sequence that prevents the wrong purchase
- Identify the intended reader and the decision they need to make.
- Write the precise subject matter and proposed claim, including the version and period.
- Determine whether a technical assessment, broader assurance programme, formal assurance engagement or certification is required.
- Confirm suitable criteria, evidence access, competence, independence and provider eligibility.
- Agree the report form, limitations, distribution and treatment of later changes.
- Plan how findings will be addressed and which conclusions require further work.
Do not let the existence of a deadline turn an unresolved report requirement into a procurement assumption. If the recipient has not confirmed what it accepts, record that uncertainty and resolve it before committing to a final deliverable. A technical audit can still begin with a useful internal decision, but its contract should not promise acceptance by an unspecified external party.
Open for AI's AI auditing service provides scoped, evidence-led technical assessment with findings and retesting. For an audit scoping discussion about the report you need, bring the recipient's request, the exact claim and the available evidence. If formal professional assurance is required, establish that route explicitly; the technical work can then be defined around a role it can properly fulfil. The board's use of AI audit reports offers further guidance on making decisions without overstating what the report establishes.
FAQ
Your questions, answered.
Is an AI audit automatically an assurance engagement?
No. It depends on the engagement terms, professional requirements and report form. A technical audit can supply evidence and findings without providing a formal professional assurance conclusion. “AI assurance” is also used broadly for a programme of confidence-building activities. Ask the provider which meaning it uses and whether the intended recipient accepts that specific deliverable. Put that meaning in the engagement terms before comparing prices or report labels.
Can a technical AI audit report be called certified AI?
That description may materially overstate the work. Certification requires its own scheme, authorised process and defined scope; a technical report does not acquire that status because tests were completed. Describe the actual assessment, version, criteria and limitations. If a certificate is required, confirm the relevant route and eligible body before commissioning work. The scope of any certificate also matters; its existence does not validate every system output.
What does limited assurance mean for an AI-related engagement?
Limited assurance is a professional engagement level, not a casual description of a short technical review. The appropriate practitioner determines whether the requested subject matter and criteria support such an engagement and what procedures and report are needed. Buyers should obtain an explanation of the specific scope and level, rather than assume limited assurance means a fixed test count or universal percentage of confidence. The report wording and intended recipients must remain consistent with the level actually commissioned.
Can a technical audit support a later formal assurance engagement?
It may, if the formal assurance practitioner determines the work is suitable for the intended purpose. Involve that practitioner early so evidence selection, documentation, access and review rights can be aligned. A completed technical report may otherwise omit information needed for reliance. The practitioner retains responsibility for its conclusion; the technical provider should not promise automatic acceptance. Agree access to supporting work before confidentiality or ownership restrictions make later review impractical.
Who should decide which assurance standard applies?
The appropriately qualified practitioner should determine the professional requirements for the subject matter, intended users and jurisdiction, in consultation with the commissioning organisation and recipient where needed. A procurement team should not choose a familiar acronym without establishing suitability. Record the determination in the engagement terms and distinguish professional standards from voluntary technical frameworks and internal evaluation criteria. The chosen route should answer the recipient’s actual decision, not merely add a familiar badge.
Does auditing an AI sustainability report generator verify sustainability claims?
Not automatically. A generator audit may examine whether selected outputs faithfully reflect source documents and follow review controls. It may not verify the authenticity of supplier documents, underlying measurements or the published sustainability statement. These are separate subjects requiring appropriate evidence and professional competence. The report and any public description must preserve that distinction. Separate source fidelity, source authenticity and underlying measurement validity in the commissioning brief.
Can an internal AI audit report be shared with customers?
Check the agreed purpose, permitted recipients, confidentiality and reliance terms first. A customer may need a different report form or additional evidence. Even where sharing is permitted, retain the scope, version, period and limitations rather than circulating a favourable excerpt alone. Later system changes may also affect whether the original conclusion remains relevant. Ask the intended customer what question it needs answered before assuming an existing report will suffice.
What should a Saudi buyer ask when a provider promises independent AI assurance?
Ask what the provider means by assurance, which professional requirements apply, who signs the conclusion and what independence relationships have been assessed. Request the precise subject matter, criteria, evidence access, report form and exclusions. Confirm whether the provider built the system or will sell remediation. A broad marketing phrase is insufficient evidence of eligibility or an acceptable independent conclusion. If a formal route is required, verify eligibility through the relevant professional or certification process.