How to Prepare for an AI Audit

Preparing for an AI audit means making the real system, its records and its responsible people accessible for a defined assessment. It does not mean manufacturing a perfect historical file or rehearsing only successful examples. Good preparation reduces avoidable delays while preserving the evidence of how the system actually operated, including known incidents, incomplete records and changes made before fieldwork.

This guide is for system owners, operations managers and audit coordinators. A hypothetical packaging-specification assistant illustrates the preparation process. The assistant drafts a structured summary from supplier specifications for human review; it does not approve manufacturing settings. The objective is an organised, truthful evidence handover, distinct from the assessor's audit checklist and eventual conclusions.

Confirm what the audit is expected to answer

Ask the sponsor to identify the decision, intended report audience and engagement form. A review supporting a controlled expansion may need different evidence from an investigation of a known discrepancy. Do not begin by uploading every document with “AI” in its title. Start with the scoped workflow and the questions the assessor must answer, then identify records that can support those questions.

Review the proposed audit scope with the process owner, technical owner and evidence custodians. Confirm systems, interfaces, languages, versions, operating period and exclusions. In the packaging example, source extraction, draft generation and human approval are different stages. If the assistant's output is later copied into another system, identify that interface even when the downstream manufacturing process is outside the assessment.

Clarify what type of report is being commissioned. A technical assessment is not automatically a certification or regulated assurance engagement. If procurement expects a particular formal opinion, resolve that expectation before fieldwork. Preparation should make the agreed assessment feasible, not create an appearance that the organisation has already passed it.

Create an honest system factsheet

Prepare a concise description of the actual workflow rather than a marketing architecture. Identify where source documents enter, how text is extracted, what reaches the model, what the assistant produces and what a person accepts or changes. Mark automated actions and manual handoffs. Include known workarounds, because the operational process may differ from the original design document.

Record available identifiers for the model service, instructions, retrieval sources, extraction component and deployment. A model name alone may not identify the tested system. If the supplier does not expose a precise underlying version, state what can be identified and what cannot. Do not fill the gap with a guessed release number.

Preparation factsheet for the hypothetical packaging assistant

Field	Useful content	Common preparation mistake
Business use	Draft structured summaries of approved supplier specifications for review.	Describing the assistant as making all production decisions.
Inputs and outputs	Source formats, extracted text, draft fields and final accepted record.	Providing only screenshots of the final interface.
Human role	Who checks units, revisions and ambiguous specification wording.	Assuming an approval button proves the review occurred.
Versions	Available component identifiers and relevant change records.	Using one current model name for the whole historical period.
Interfaces	Supplier document store, review queue and any downstream transfer.	Omitting a manual export because it is not an API.
Known limits	Unsupported formats, missing records and supplier access restrictions.	Hiding gaps until the assessor discovers them.

The factsheet should point to evidence rather than duplicate it. Keep it current through a named owner and record changes during preparation. A clear limitation is useful information: it allows the assessor to adjust procedures or reporting expectations. An inaccurate claim of completeness creates unnecessary rework and can undermine confidence in the evidence pack.

Inventory existing evidence before requesting new documents

Identify what already exists, where it is held, who controls it and which period it covers. Relevant material may include approved specifications, change records, permissions, source-output pairs, review events, incident records and monitoring responses. Separate documents describing intended controls from records showing operation. A recently written procedure can describe the current design without proving that it governed the earlier period.

The GAO Government Auditing Standards provide a public professional reference for evidence and audit documentation in their applicable context. They are not automatically binding on a Saudi commercial technical assessment. The practical preparation principle here is narrower: retain a traceable basis for the records and explanations supplied, and let the assessor evaluate their sufficiency for the agreed engagement.

Original evidence-readiness inventory

Evidence group	Owner to involve	Preparation check	Gap to disclose
Source specifications	Document or supplier-data owner	Identify approval, revision and effective context.	Historical revisions unavailable or source authority unclear.
Processing records	Integration owner	Locate submitted input and relevant transformation evidence.	Only final output retained, with no intermediate trace.
Generated drafts	Application owner	Preserve selected identifiers and available configuration context.	Outputs overwritten by later human edits.
Review decisions	Process owner	Locate approval, correction and referral events.	Review asserted but not recorded in the relevant period.
Change history	Release owner	Map changes to affected deployments and dates.	Unrecorded instruction or source changes.
Incidents and exceptions	Operations or risk owner	Preserve reports, investigation records and responses.	Informal messages are the only available account.

Mark each item available, partially available, unavailable or restricted, with a short reason. This is a readiness status, not a control assessment. An available document may be irrelevant or insufficient, while a restricted source may still be inspectable through an agreed arrangement. Do not translate the inventory into a “compliance percentage”.

Preserve historical truth while improving current documentation

It is reasonable to document an undocumented process before an audit. Label the document with its actual creation and approval dates, explain what it describes and preserve any supporting historical records. Do not backdate it or present a newly written procedure as proof that everyone followed it months earlier. The assessor may examine both the historical gap and the current improvement.

In the packaging example, a newly created rule for handling ambiguous units may be useful. It does not establish that earlier drafts were checked under that rule. Preserve the original source and draft records where available, identify when the new rule took effect and show how staff were informed. This allows the audit to distinguish past operation from the revised process.

Do not delete inconvenient records during preparation or replace an error with a corrected output without retaining the relationship. Legitimate corrections should have a clear history. If routine retention has already removed evidence, record that fact and the applicable policy. A truthful gap can be evaluated; a reconstructed record disguised as contemporaneous evidence cannot be relied upon in the same way.

Arrange access and handling before fieldwork

Identify the people who can approve access, the systems involved and the permitted evidence channels. An audit coordinator may organise requests without having authority to release every record. Confirm personal-data, confidentiality and supplier conditions with the responsible owners. Use evidence-handling arrangements to define collection, masking, storage, transfer and retention.

Prepare authorised test accounts with the roles needed for the agreed procedures. Avoid giving broad administrator access merely to reduce setup effort. If the assessment needs to compare different permission levels, arrange distinct controlled roles and record their purpose. Production actions should remain within the explicit rules of engagement; preparation does not authorise the assessor to send messages, change records or trigger real transactions without the necessary permission.

Test access before the working session. Confirm that links resolve for the intended reviewer, not only for the owner who created them. Check that shared folders do not expose unrelated material through inherited permissions. Where export is restricted, arrange supervised inspection and explain what can be recorded. The assessor should know the limitation before a procedure depends on unavailable access.

Prepare populations and examples without curating away failures

If the assessor needs an operational population, provide the extraction method, eligibility rules, period and reconciliation information. Do not supply only successful or completed outputs unless that is the explicitly agreed population. Include failed processing, pending reviews and cancelled cases according to the scope rules. Let the assessor determine the selection method rather than choosing the most favourable examples.

A few masked examples can help explain the workflow during preparation. Label them as illustrative and state who selected them. They are not a substitute for the audit sample. Provide an ordinary case, an exception and a case with incomplete information where available. The purpose is to reveal how the process works, including its limits, not to demonstrate a perfect performance record.

For Arabic and English specifications, identify source language, mixed-language content and available reviewers. Preserve units, negation, qualifiers and revision references in masked examples. If translation is necessary, label it as a derivative and retain access to the original meaning. Do not let a translated, simplified example conceal the difficulty the live assistant actually encountered.

Assign people to questions, not just meeting invitations

Map likely questions to the people who can answer them and the records they control. The process owner explains intended business decisions; the technical owner explains transformations and versions; the source owner confirms document authority; the reviewer explains how exceptions are handled. One enthusiastic coordinator cannot reliably substitute for all of these roles.

Brief participants on the purpose and scope of the audit. Encourage factual explanations, uncertainty where appropriate and references to supporting records. Do not coach staff to give identical reassuring answers or conceal workarounds. Differences between accounts can reveal a process gap that the organisation needs to understand. A rehearsed narrative may delay that understanding rather than improve readiness.

Arrange specialist availability where the assessment requires it. A packaging-specification term may need the responsible domain expert; a statistical claim may need an analyst; an Arabic passage may need a competent language reviewer. Identify substitutes and escalation routes so fieldwork does not depend entirely on one person's calendar. The plan should make expertise available without implying that every specialist approves the audit conclusion.

Use a gap log to make uncertainty actionable

Record missing or restricted evidence before fieldwork, with the affected question, owner, potential alternative and decision needed. Distinguish “not yet located” from “never recorded” and “not permitted to export”. These conditions require different responses. A broad label such as “documentation pending” can hide a limitation that cannot be solved by writing another document.

Illustrative preparation gap log

Gap	What can be done honestly	What must not be claimed
Historical draft overwritten	Seek controlled backups or related records; disclose if the original cannot be recovered.	A newly generated draft is the original historical output.
Review took place outside the system	Locate contemporaneous records and explain their limits.	A later recollection proves every review occurred.
Supplier version not exposed	Record available service identifiers and request relevant supplier information.	A precise underlying model release has been verified.
Source revision unclear	Ask the authorised source owner to identify what can be established.	The current document necessarily governed the earlier output.
Restricted personal information	Arrange authorised masking or controlled inspection where suitable.	The restriction can be bypassed because the request concerns an audit.

Share material gaps with the assessor and scope owner early. They can decide whether alternative procedures are sufficient, the period should change or the conclusion must be narrower. Preparation succeeds when those decisions are explicit. It does not require every historical limitation to disappear before work can begin.

Run a truthful walkthrough

Use a controlled case to show the actual path from source to accepted record. Explain each handoff and identify the evidence it creates. For the packaging assistant, show how the source revision is selected, how a draft field is compared with the source and what happens when a unit or qualification is ambiguous. Use an authorised environment and avoid real consequential changes merely to demonstrate the interface.

Record where the walkthrough differs from ordinary operation. A training environment may use simplified data or different permissions. A case prepared in advance may not represent the full population. These facts do not make the walkthrough useless; they define its role as orientation. The assessor's independently selected procedures provide a different form of evidence.

Use the session to find missing links and unclear responsibilities. If the team cannot locate the source corresponding to the displayed draft, record that gap rather than switching silently to another case. If an employee explains an undocumented workaround, preserve the explanation and seek corroborating records. Preparation is an opportunity to expose the real process to examination.

Manage changes while the audit is under way

Agree a change-notification process for models, instructions, sources, permissions and interfaces. A complete freeze may be impractical, but unannounced changes can make evidence difficult to interpret. Record what changed, when, why and which deployments were affected. The assessor can then decide whether results require separate treatment or additional procedures.

Continue necessary operational improvements with appropriate authorisation, while preserving the distinction between tested historical conditions and later changes. Do not delay a needed protective action solely to keep the audit environment unchanged. Notify the assessor, retain relevant evidence and explain the effect on scope. The decision to contain exposure belongs in the operational process, with the audit adapting transparently.

Maintain one controlled request log during fieldwork. Record the request, purpose, owner, response and unresolved limitations. Avoid sending multiple inconsistent copies through informal channels. The log should distinguish evidence supplied from questions answered verbally and from items still pending. It helps both sides manage work without treating delivery of a file as proof that the file is adequate.

Test the evidence handover with one complete request

Before fieldwork, rehearse the logistics of one legitimate evidence request without pre-judging its audit result. For example, ask the coordinator to locate a masked packaging-specification case, identify the applicable source revision, retrieve the generated draft and locate the human review record. The exercise tests whether the organisation can supply the requested material coherently, not whether the assistant performs correctly.

Record how long each dependency takes and where responsibility becomes unclear. A source owner may identify the document quickly while the application team cannot retrieve the unedited draft. That distinction matters for planning. Do not conceal the missing item by substituting a different case unless the substitution is explicitly recorded and the assessor decides whether it is suitable for the intended procedure.

Check the delivered package from the recipient's perspective. Can the reviewer understand the index, distinguish originals from derivatives and relate the records to the same case? Are version references and timezones clear? Does access reveal only the intended material? A package can contain every requested filename and still be unusable because the relationships are missing.

Use the result to improve coordination and disclose persistent limitations. If the review record was never created, the coordinator should record that fact rather than ask someone to sign a retrospective approval as though it occurred at the time. A current explanation can be useful, but it must carry its actual date and evidential limit. This preserves the difference between helping the assessor understand the process and manufacturing the evidence the process should have produced.

Finally, confirm the contact for urgent questions during fieldwork and the person authorised to decide on scope or access changes. Administrative readiness includes being able to resolve a genuine blocker without informal permission stretching. A clear route for decisions reduces delay while keeping the assessment within its agreed authority.

Know what ready means

The organisation is ready to begin when the scope and decision are understood, responsible people are available, authorised access is workable, existing evidence is indexed and known gaps are disclosed. That does not mean every control is effective. It means the assessment can proceed with an honest basis for its methods and limitations.

Ask the assessor what will be retained in the working papers and how factual review of the report will occur. Plan for management responses and evidence-led follow-up without assuming a favourable result. OpenForAI's AI auditing service can be discussed around these concrete preparation needs. Discuss how to prepare your system and evidence for an AI audit.