What is AI Auditing?

AI auditing is a structured examination of an AI system against stated criteria, using evidence that can support a bounded conclusion. The useful question is not whether a system deserves a reassuring label. It is whether a particular claim about its behaviour, controls or operation is supported well enough for a particular decision. A buyer may need to decide whether to release a new workflow, restrict an existing one, require supplier changes or commission further testing.

This guide is for business owners, technology leaders and assurance teams who need to understand what they are buying. An audit should make the connection between a requirement, the evidence examined, the result observed and the conclusion reached visible. It should also make missing evidence visible. The result is useful only when its reader can distinguish a control that worked during testing from a control that was merely described in a policy.

The object of an AI audit is the working system

An AI system includes more than a model. Inputs are collected and transformed, a model produces an output, software presents or acts on that output, people interpret it, and records are retained or discarded. Each part can affect the business result. A technically strong model can sit inside a workflow that sends information to the wrong person. An accurate recommendation can become unsafe when a user assumes it is an authorised decision.

For that reason, the audit boundary needs a named use case, users, permitted actions, data sources, environments and versions. It also needs a period: a point-in-time configuration review answers a different question from a review of operation over several weeks. A report that says only “the AI was audited” conceals these distinctions. The buyer should be able to identify what changed after the evidence was collected and whether that change affects reliance on the conclusion.

The NIST ARIA evaluation planning manual combines model, red-team and user testing when designing holistic evaluations. That is useful context for choosing complementary evidence. It does not turn any combination of tests into certification, and the audit still needs its own agreed purpose, criteria and reporting boundary.

Start with a claim that can be examined

“Our assistant is trustworthy” is too broad to test directly. “The assistant marks uncertain image descriptions for human review before they enter the case record” is more useful. It names a behaviour, a workflow transition and a decision point. The auditor can ask what counts as uncertainty, where the review state is enforced, which records demonstrate operation and whether anyone can bypass the step.

A claim can concern performance, access, approval, record keeping or governance. Different claims require different evidence. A sample of correct answers cannot establish that access permissions are enforced. A permissions diagram cannot establish that the system produces accurate Arabic descriptions. A signed policy cannot establish that staff actually reviewed an output before using it. Breaking a broad claim into testable statements prevents impressive evidence in one area from concealing an unanswered question in another.

Criteria should be agreed before results are known. They may come from applicable requirements, approved internal policies, contractual commitments or a clearly labelled evaluation rule developed for the engagement. Record the source and owner of each criterion. An auditor should not quietly replace a failed criterion with an easier one because the system performed poorly. If the original criterion is inappropriate, revise it transparently and explain how that affects the conclusion.

A worked example: describing freight-damage photographs

Consider a hypothetical freight operator using AI to draft descriptions of damaged packaging from photographs. The assistant helps a human reviewer describe visible features. It does not assess the value of damage, assign liability, approve compensation or determine the cause of an incident. Those exclusions matter: a convincing description of a dent does not establish when the dent occurred or who caused it.

The proposed audit covers one deployed application version, two supported languages and the transfer of a reviewed description into a case-management record. It examines whether descriptions stay within visible evidence, whether uncertainty reaches the reviewer and whether the final record preserves the distinction between AI draft and human amendment. Customer identifiers are masked in the evaluation set, and authorised staff control access to the underlying photographs.

Illustrative evidence-to-conclusion map for the photo-description workflow

Claim	Procedure and evidence	Permitted conclusion
Descriptions do not invent damage absent from the photograph	Compare selected drafts with photographs using agreed reviewer criteria; record disputed cases separately	Observed support or exceptions in the selected cases, with selection limits
Uncertain drafts require review	Inspect the enforced state transition and test controlled uncertain cases	Whether the tested transition required review in that configuration
The final record identifies human amendments	Trace draft, reviewer identity, amendment and saved record for selected completed cases	Whether amendment provenance was available for those records
The assistant determines liability correctly	No procedure: liability determination is outside the agreed use and scope	No conclusion on liability, compensation or valuation

The table is an original planning example, not a regulatory checklist. Its value is the discipline of matching every conclusion to a procedure. If the team cannot retrieve the human amendment record, it should not write that amendments were absent. It should first distinguish a missing export, an incomplete retention process and an actual failure to record amendments. The eventual finding may concern evidence availability rather than the original claim.

What auditors do with documents, tests and interviews

Documents establish the intended design and the commitments against which it may be examined. Configuration and implementation evidence help explain whether the intended control exists. Executed tests show behaviour under specified conditions. Operating records can show how the control functioned during a period. Interviews explain responsibilities and can identify contradictions, but a confident explanation does not replace a missing record.

These sources are strongest when they corroborate each other. Suppose a procedure says reviewers must approve all descriptions, the interface displays an approval button and selected saved records contain reviewer identities. The auditor still needs to establish whether saving is possible without approval and whether the identity relates to the exact text saved. The separate pieces are necessary context; their relationship determines what they prove.

Evidence can also contradict a claim. A documented restriction may be bypassed in a controlled test, or a retained record may show an unapproved output entered the workflow. Preserve that contradictory evidence rather than averaging it away with successful demonstrations. The engagement should explain how disputed observations are reproduced, how alternative explanations are examined and who reviews the reasoning before the report is issued.

Design effectiveness and operating effectiveness answer different questions

A design review asks whether a proposed control could reasonably address a defined risk if implemented and used as intended. An implementation check asks whether the control is present in the examined environment. An operating review asks whether it functioned in the selected events and period. These are related questions, but none can be substituted for the others without explanation.

In the photograph example, requiring a reviewer field in the data model may be a sensible design. Seeing that field in a test environment confirms only part of implementation. Tracing real completed cases may show that a service account populated it automatically, defeating its intended meaning. Conversely, missing historical records may prevent an operating conclusion even when a newly tested control now works. The report should name the evidence gap and the period affected.

This distinction also changes the engagement deliverable. A pre-launch audit may legitimately focus on design, controlled testing and release conditions. A post-launch audit can examine operating evidence, but only where records exist and are reliable enough. The practical approach to scoping an AI audit should state these choices before the buyer compares providers or assumes that every audit supplies the same depth.

How to read an audit result without overstating it

Imagine an illustrative evaluation of 120 selected photo cases: 80 routine cases chosen from an agreed operating population and 40 deliberately challenging cases involving glare, partial views or poor focus. The two groups answer different questions. Routine selection may support a population inference only if its sampling assumptions justify that inference. The challenge set helps expose weaknesses under selected conditions; its failure rate is not an estimate of everyday prevalence.

Suppose 7 of the challenge cases contain unsupported descriptions, while reviewers disagree on 3 additional cases. The report should show 7 confirmed exceptions and 3 unresolved assessments, with their denominators and criteria. It should not silently classify disagreement as success, combine unlike groups into a universal accuracy score or claim that the remaining photographs prove the system safe. Another reviewer, different images or a changed version may produce different evidence.

Three statements about the same hypothetical audit

Statement	Assessment	Better wording
The system is 100% reliable	Unsupported: neither the selected population nor every possible condition was examined	Describe the observed results and material untested conditions
Seven failures prove a 17.5% production failure rate	Unsupported: 7 of 40 deliberately difficult cases is a challenge-set result	Seven confirmed exceptions occurred in the 40 selected challenge cases
The approval control worked for all tested transitions	Potentially supportable when the transitions, configuration and evidence are named	Limit the claim to those tests and disclose unexamined paths

Results become more useful when the reader can see where uncertainty remains. AI audit sampling explains how selection affects interpretation. A small number of carefully chosen tests can reveal a serious weakness without estimating its frequency. Equally, a large dataset can be misleading when duplicated records, leakage or an unsuitable reference standard undermine what is being measured.

What belongs in a finding

A finding connects an observed condition to a criterion and an implication. For example: an uncertain description could be saved without the required reviewer confirmation in a specified tested configuration. The evidence should identify the relevant test, the expected behaviour, the observed transition and the destination record. The implication is that an unreviewed description could enter the workflow; it is not automatically proof of a paid compensation error.

Keep the cause separate until it is supported. A missing server-side check, stale interface state and a misconfigured integration may produce similar symptoms. The auditor can describe a hypothesis and the additional evidence needed to distinguish it. Management can then assign a remediation owner without pretending the root cause is already settled. Severity should reflect consequence, exposure and uncertainty, not the dramatic appearance of a screenshot.

The report should also show management's response, the agreed next step and what would count as closure. An accepted recommendation is not a completed fix. A developer's statement that the code changed is not retest evidence. Closure requires a defined procedure addressing the original condition and relevant neighbouring behaviour. The guide to AI audit retesting explains why the version and evidence of the corrected workflow matter.

Who can perform an AI audit?

The team needs competence matched to the claim. Statistical evaluation, application security, data governance, business operations and sector requirements are different disciplines. One person may cover several, but the proposal should explain who is responsible for each material judgement and who reviews it. A general AI credential does not establish competence in every model family or every regulated workflow.

Independence must also be specific. A team that designed a control has valuable knowledge, but assessing its own work creates a self-review threat. The engagement should disclose prior roles, commercial incentives and reporting arrangements, then describe any safeguards and remaining limits. A separate company name or a different project manager does not by itself resolve the issue. Read AI audit independence and conflicts before relying on an “independent” label.

For a Saudi organisation, the applicable obligations depend on its entity, activity and data processing. An audit should record who determined applicability and which criteria the engagement actually tests. It should not infer comprehensive legal compliance from a generic technical test set. Where legal interpretation or a regulated opinion is needed, the engagement must identify the appropriately qualified role and distinguish that work from the technical assessment.

Audit, evaluation, assurance and certification

An evaluation can measure a specific capability without assessing the wider control environment. An audit can use evaluations alongside other evidence to address agreed questions. Assurance can describe a broader confidence-building programme, while a formal professional assurance engagement has defined requirements and a specific form of conclusion. Certification involves its own scheme, scope and authorised certification process. These terms should not be interchangeable in procurement documents.

A technical audit report can be valuable without being a certification or a statutory opinion. Its value lies in a defensible account of what was examined and what follows from the evidence. If a buyer needs a report for a lender, regulator or external customer, establish the recipient's required form before commissioning the work. AI audit versus AI assurance provides the questions needed to avoid buying a deliverable that the intended reader cannot use.

Reconstruct one claim from beginning to end

A buyer can test the usefulness of a proposed audit approach by asking the provider to reconstruct one example claim. For the photograph assistant, begin with the source image and its authorised use. Identify the transformation applied before the model sees it, the generated description, any uncertainty flag, the review decision and the final stored text. At each transition, ask what record would demonstrate that the transition occurred as described. This exposes missing evidence before the engagement becomes a collection of disconnected tests.

Then introduce a controlled variation. Keep the image constant while changing only the permitted reviewer role, or keep the role constant while using an image for which the agreed reference decision is uncertain. The purpose is to understand the control, not to manufacture a dramatic failure. A change in output may reveal sensitivity to the condition, while a rejected transition may demonstrate enforcement. The procedure needs to record what was held constant and what changed so that the observation is interpretable.

Finally, ask a second competent reviewer to follow the reasoning using the retained evidence. They need not reproduce every internal model computation to assess whether the conclusion follows. They do need enough context to distinguish a model response from a saved record, a reviewer action from an automatic field and a test result from an assumption. If that reconstruction is impossible, the immediate improvement may be better evidence capture rather than more test cases.

This exercise also helps decide when an audit cannot answer the desired question. A supplier may expose only an interface, the historical source version may be missing or the reference judgement may remain disputed. Those conditions should shape the conclusion and next decision. They do not make the system automatically defective, but they prevent the buyer from treating an unexamined claim as established. The practical value of an audit is often that it makes this boundary visible before a consequential commitment.

What to ask for before commissioning work

Ask for a written purpose, system boundary, criteria, evidence plan, access model, deliverables and limits. Require clarity about languages, versions, production access, authorised security exercises, handling of sensitive evidence and retesting. Ask what happens if a supplier cannot provide material evidence or the system changes midway through testing. These are scope decisions, not details to discover after the fee has been spent.

A useful handover includes a readable report and traceable supporting work, with enough information for responsible teams to understand and reproduce relevant findings within the agreed access constraints. It should identify decisions required now, remaining questions and the evidence needed to revisit them. AI audit report structure explains how conclusions, findings and limitations should fit together without burying the decision in technical attachments.

Open for AI's AI auditing services are scoped around evidence-led technical assessment, findings and retesting. To begin an audit scoping discussion for your AI workflow, bring a masked description of the use case, its users and actions, the version to examine and the decision the report must support. That information makes it possible to define an audit whose conclusion is useful because its boundaries are clear.