AI Audit Evidence Handling

AI audit evidence handling controls who may collect evidence, what is collected, how it is protected and how a reviewer can understand its history. It is different from simply saving a copy of a chat transcript. Inputs, outputs, retrieved documents, tool events and human decisions can contain sensitive information, change over time or lose context when exported. A useful handling process preserves enough support for the audit while avoiding unnecessary duplication and access.

This guide is for audit managers, security teams and evidence custodians. It uses a hypothetical tenancy-support enquiry assistant to illustrate collection and custody. The assistant handles service enquiries; the example does not determine tenancy rights or provide legal advice. The proposed records are practical internal designs, not a claim that a particular file will be admissible in court or that a technical audit automatically meets every regulatory requirement.

Establish authority and purpose before collecting records

Identify the audit objective that requires each evidence category. An assessor examining whether enquiries were escalated may need event identifiers, timestamps, routing decisions and selected content context. They may not need every attachment or a full identity document. Link the requested material to the question it supports. “It might be useful later” is a weak reason to create another sensitive archive.

Confirm who can authorise access and whether the proposed use is within the engagement and organisational permissions. A system owner may control the application without having authority to disclose all personal information inside it. Supplier confidentiality, contractual restrictions and applicable data obligations may require additional review. Address those conditions before transfer, rather than assuming an audit label creates unrestricted access.

For Saudi personal-data questions, identify the relevant processing activities, parties and data flows with the responsible privacy or legal owner. The AI audit guide to PDPL addresses that applicability and evidence work. This handling guide does not assume that all Saudi data has the same residency rule or that a particular hosting region makes processing compliant. Record the actual authorised arrangement.

NIST SP 800-86 discusses digital forensic techniques in an IT incident-response context and distinguishes that guidance from legal advice. Its treatment of digital evidence is a useful reference for preservation questions. An ordinary AI audit still needs its own proportionate handling plan, and any legal evidential requirements should be resolved by qualified owners for the specific matter.

Create an evidence inventory with sensitivity and custody fields

Assign an evidence identifier before material begins circulating. Record the source, collector, collection time, relevant system and version, purpose, sensitivity, location and permitted recipients. Identify whether the item is an original export, a transformed copy, a screenshot, an interview note or a derived analysis. These types have different limitations and should not appear interchangeable in the index.

For the tenancy-support example, a conversation export may include names, addresses and attachments that are not needed to examine routing. A masked case extract can preserve the relevant sequence while reducing exposure, provided the masking process is documented and does not remove the feature being tested. If location wording affected routing, removing it may change the evidence; use a controlled alternative that preserves the necessary property.

Proposed evidence inventory fields

Field	What to record	Review question
Identity and source	Stable evidence ID, source system, export method and source owner	Can the item be related to an authorised source?
Collection context	Collector, time, timezone, query conditions and available version identifiers	What exactly was captured, and when?
Scope and purpose	Audit question, population or selected case references	Why is this material necessary?
Classification and access	Sensitivity, permitted roles and controlled storage location	Who may inspect or receive it?
Integrity and lineage	Recorded digest where appropriate, parent item and transformations	Can changes and derived copies be distinguished?
Retention and disposition	Applicable rule, review point, preservation restriction and disposal owner	When and how will authorised disposal be considered?

The inventory is a map, not a second copy of the sensitive evidence. Use stable references and minimal descriptions. Restrict access to raw items separately from access to the index. An executive reviewing the findings may need the meaning and significance of an observation without access to the underlying tenant conversation.

Preserve the original capture and label derived material

Where authorised and necessary, preserve a controlled original capture and create working derivatives for analysis. Do not redact or translate the only retained version and then describe it as the original. Record what changed, why, by whom and with which method. A redacted copy, a translated transcript and a table of extracted events are distinct evidence products with different strengths and limitations.

For the hypothetical enquiry, the original export may contain the complete event sequence. A masked derivative removes identifiers not needed for the test. A translation supports a reviewer who does not read the source language. An analysis table records the assessed routing and escalation events. Each derivative should point back to its parent, while access to the parent remains controlled. The reviewer should know which conclusions depend on a transformation.

Illustrative evidence lineage for one enquiry

Item	Role	Transformation record	Important limit
EV-21	Authorised source export in restricted storage	Collection method and integrity record	The export may still require a completeness check against the source system.
EV-21-M	Masked working copy	Fields removed or substituted and reviewer approval	Masking must not erase context relevant to routing.
EV-21-T	Reviewed translation of relevant passages	Source language, translator or method and unresolved ambiguity	A translation is not the source-language original.
EV-21-A	Event analysis derived from the controlled material	Extraction rules, case references and adjudication	The analysis supports an interpretation, not independent source authenticity.

Keep hidden metadata in mind. A document may contain comments, tracked changes, embedded objects or author information. An image may retain location or device metadata. Inspect the form actually being shared and apply an authorised minimisation process. Renaming a file or covering text visually may leave recoverable content; confirm that the intended redaction is effective in the released derivative.

Use hashes for integrity without overstating their meaning

A cryptographic digest can help identify whether a file's bytes changed after the digest was recorded. Record the algorithm, value, time and the item to which it applies in a trusted location. Verify the digest after transfer where appropriate. If the content changes legitimately through redaction or conversion, create a new derivative with its own record rather than treating the different digest as unexplained corruption.

A matching digest does not prove that the source was truthful, that the export was complete or that collection was authorised. It also does not prove that the recorded timestamp accurately reflects the underlying event. Integrity of a captured file and authenticity of the represented event are different questions. A fabricated file can be hashed just as easily as a genuine one.

Protect the integrity record itself. If the same person can replace both the evidence and the recorded digest without trace, the comparison offers limited protection against that change. Use access controls and change history appropriate to the engagement. Do not describe a digest alone as a complete chain of custody, digital signature or guarantee of legal admissibility.

Record custody events at meaningful handoffs

A custody record explains who held or accessed the material and what happened to it. Record collection, transfer, receipt, transformation, restricted disclosure and authorised disposal where relevant. The level of detail should fit the sensitivity and purpose. Routine read access may be captured by repository logs, while a transfer to a separate organisation may need an explicit handover record.

Original transfer-and-receipt record

Handoff field	Required entry
Material	Evidence IDs, versions and file inventory, with restricted details kept separately.
Authority and purpose	Approved recipient, permitted use and relevant engagement reference.
Transfer method	Authorised channel and access duration; credentials are not included in the record.
Sender and recipient	Actual responsible identities or controlled role references.
Timing and integrity	Dispatch and receipt times, timezone and integrity verification where used.
Exceptions	Missing items, unreadable files, failed checks or changed conditions and their resolution.

Do not repair a custody gap by inventing an event. If a historical transfer was not logged, record the gap, seek corroborating information and assess its effect on reliance. A later statement may help explain what probably happened, but it does not become a contemporaneous transfer record. Transparency about the limitation is more useful than a falsely complete table.

Collect AI context without collecting everything

An AI output can depend on instructions, retrieved content, tool results, service configuration and human edits. Determine which elements are necessary to interpret the specific observation. For a routing finding, the submitted enquiry and relevant routing rules may matter more than unrelated conversation history. For an unauthorised retrieval concern, access context and source identifiers may be essential. Tailor collection to the tested assertion.

Distinguish the user's original text, the input actually submitted to the model, the generated output and the final text accepted by a person. These may differ. If only the final text is preserved, the assessor may be unable to attribute a discrepancy to the model rather than an edit. Record that limitation rather than reconstructing a fictional original prompt from memory.

Be careful with secrets and privileged information in logs. Access tokens, credentials and unrelated personal material should not enter the working set merely because an export includes them. Use an authorised filtering or restricted-inspection approach that preserves the evidence needed. If an unanticipated sensitive item is discovered, limit further exposure and involve the responsible custodian rather than circulating it to demonstrate the problem.

Handle time, ordering and changing sources explicitly

Record timezone and the meaning of each timestamp. Event occurrence, service receipt, export creation and auditor collection are different moments. Clock differences can affect sequence interpretation, especially when events cross systems. Preserve available timing metadata and state uncertainty where clocks or offsets cannot be reconciled. A sorted spreadsheet does not establish a causal sequence by itself.

Capture source revisions relevant to the observation. A knowledge page or policy may change after a model response was generated. Comparing the response with today's version can misstate what information was available at the time. Where historical content is unavailable, say so and narrow the conclusion. A current screenshot is evidence of current presentation, not proof of a past state.

For dynamic dashboards or short-retention logs, agree a timely collection plan before fieldwork depends on them. Preserve the query, filters, observation time and exported result where authorised. If a dashboard aggregates away individual cases, determine whether it can support the audit question. A total count may be useful for reconciliation while remaining insufficient to reconstruct a selected enquiry.

Set access and transfer controls that fit the engagement

Use approved storage and transfer arrangements, named access and time-bounded permissions where appropriate. Separate broad report readership from restricted evidence access. Review permissions when staff change roles or the engagement ends. A temporary external reviewer should not retain indefinite access because removing it was omitted from the handover checklist.

Test the recipient's ability to access the intended material without exposing more than necessary. Broken links can delay review, while overly broad folder sharing can reveal unrelated evidence. Check inheritance, downloadable copies and external sharing settings in the actual repository. The handling plan should describe what the organisation has authorised, not merely state that the files are “secure”.

Do not upload audit evidence to an unapproved AI summariser, translation service or personal file-sharing account for convenience. If an authorised tool processes evidence, document its role, permitted data and relevant handling conditions. A generated summary is a derivative that needs verification; it should not replace the controlled source or become an unsupported factual record.

Plan retention, preservation and disposal together

Determine retention from the applicable engagement, professional, contractual, legal and organisational requirements. This guide does not establish a universal number of years. Separate the evidence needed to support final conclusions from temporary extracts and redundant copies. Assign a custodian responsible for reviewing retention and carrying out authorised disposal.

A preservation requirement can suspend ordinary deletion for relevant material. The responsible legal or records owner should determine its scope and duration. Do not infer that every audit requires indefinite retention, or that ordinary deletion can proceed despite a known preservation instruction. Record the decision and apply it consistently to controlled originals, derivatives and authorised copies.

Disposal should be verifiable to the extent appropriate for the storage arrangements. Record what was disposed of, by whom, when and under which authority, together with limitations such as managed backups that follow a separate lifecycle. Do not promise immediate erasure from every backup if the architecture does not support it. Close access and reconcile copies as part of the engagement's completion.

Use controlled inspection when export is inappropriate

Sometimes an authorised reviewer can inspect evidence within the owner's environment but cannot receive a copy. That arrangement can be useful if the procedure and record are designed around it. Agree which items the assessor may select, what source context is visible, whether notes or screenshots are permitted and how the observation will be confirmed. A supervised demonstration chosen entirely by the owner provides different evidence from assessor-selected inspection.

For the tenancy-support example, the custodian might expose selected case records in a restricted workspace while allowing only masked identifiers and procedural observations into the working papers. The assessor should record the selection method, inspection time, visible fields, source version and limitations. If no preserved copy can be retained, explain how a later reviewer can corroborate the observation and what cannot be reconstructed after the source changes.

Do not use a screen recording to bypass an agreed no-export restriction. Recording can itself create a new sensitive copy, including information visible outside the selected case. Obtain the relevant permission and minimise the captured material if recording is justified. The restricted-access audit approach should address these constraints openly rather than treating them as a technical inconvenience to work around.

Confirm the arrangements in the audit scope before relying on this method. A controlled inspection may be sufficient for one question and insufficient for another. Seeing a selected approval event can support that observation, while estimating the frequency of missing approvals may require a reconciled population and a different access design. State the resulting inference boundary instead of describing all supervised access as equivalent.

If the inspection reveals an unintended disclosure, stop unnecessary collection and involve the responsible custodian. Record enough to explain the evidence issue without spreading the exposed material. The organisation's authorised incident and privacy processes should determine the response, including any further preservation or notification decisions. The auditor should preserve a truthful account of what was observed while avoiding an expanding archive of the very information that should have been restricted.

At completion, reconcile temporary access, locally created derivatives and permitted notes against the handling agreement. Obtain confirmation of actions that another custodian controls rather than claiming to have personally deleted material in systems the assessor cannot inspect. A clear handover record should distinguish completed actions, third-party confirmations and any remaining retention or access conditions.

Respond to evidence problems without overstating the conclusion

If a file fails an integrity check, preserve the received item, restrict reliance and investigate the discrepancy. It may reflect a legitimate conversion, transfer error or unauthorised change. The cause should be established where possible. A mismatch is a reason for examination, not automatic proof of malicious tampering.

If an item is incomplete, unavailable or no longer attributable to its source, record the effect on the relevant audit question. Seek alternative evidence where feasible and keep the limitation visible in the working papers and the report. Do not treat an evidence gap as a clean result, and do not turn it into a proven system failure without support.

For an AI audit scoping discussion, prepare an evidence inventory outline, the proposed access model and a masked example of the records available. Discuss authorised evidence collection and handling for your AI audit.