AI Audit for Generative AI Systems

An audit of a generative AI system examines how inputs become generated assets, how those assets are evaluated and edited, and which version is ultimately used. The subject may be images, audio, video, documents or mixed-media output. A good result is not established by visual polish alone: the asset must meet its intended purpose, preserve required facts and pass the controls relevant to its use.

This guide focuses on a hypothetical catalogue-image workflow that generates backgrounds around approved product photographs. The product itself must retain its observable characteristics. The example is a ceramic storage jar; the workflow may change the surrounding scene but must not invent a different lid, capacity marking or included accessory. It is a controlled illustration, not a claim about a completed client audit.

The NIST report on synthetic-content transparency examines provenance, labelling and detection approaches. Those are relevant evidence tools, not substitutes for reviewing the content and its intended use. A scoped AI audit should state which modalities, workflow stages and claims it actually examines.

Define the permitted transformation

Start with a clear creative and factual brief. In the catalogue example, background generation is permitted while the product's shape, visible features and approved markings must remain faithful to the source. The audit needs that distinction before evaluating outputs. Otherwise, a reviewer may accept an attractive redesign that the business never intended to depict.

Identify which aspects are factual, which are creative and which remain uncertain. Colour can be affected by lighting; a generated shadow may imply a different shape; an added accessory can suggest it is included with the product. The responsible product and content owners should establish the acceptance criteria and the evidence that resolves disagreements.

Define the use boundary. An internal concept image and a published catalogue image create different reliance. A generated scene approved for ideation should not automatically become a sales asset. The audit should follow the asset's actual destination and the decision authorising that use, rather than relying on its initial label.

Inventory inputs and the authority to use them

List the source photographs, reference images, brand elements, text and instructions supplied to the generator. Record their owners, approved uses and relevant restrictions. The audit should examine the organisation's evidence and decision process without claiming that a technical test alone establishes copyright ownership or every legal permission.

Distinguish the product photograph from stylistic references. A reference supplied to suggest lighting should not silently become a source for a new product feature. Preserve the intended role of each input and the configuration used. If a provider-controlled model's training history is unavailable, record that limitation rather than inventing a complete provenance story.

Review data minimisation for uploaded assets. Source files may contain people, background details, internal metadata or commercially sensitive information that the generation task does not require. Establish who authorises upload and which service settings apply. The evidence handling guide also applies to copies retained for the audit.

Trace the asset through generation, selection and editing

A generative workflow often creates several candidates from one brief, then selects and edits one. The audit trail should preserve the relationship between the source, generation request, candidate, edit, approval and published rendition. A filename alone may be insufficient if it can be overwritten or reused for different assets.

Record available model and application identifiers, instructions, relevant settings and input versions. A seed, where supported, can be useful evidence, but should not be described as a guarantee of identical future output across changed software or provider infrastructure. Preserve the actual asset examined, not only the instructions that produced it.

Original asset-lineage record for the catalogue-background example

Stage	Evidence	Audit question
Approved input	Source asset identifier, version and permitted transformation.	Which product facts must remain unchanged?
Generation	Request identifier, input links, configuration and produced candidates.	Can the selected output be linked to its actual inputs?
Selection	Chosen candidate and selection rationale.	Was creative preference separated from factual acceptance?
Editing	Derived asset identifiers and material changes.	Did later work alter a characteristic already reviewed?
Approval	Reviewer, criteria and exact asset version.	What content and intended use were approved?
Publication or handoff	Delivered rendition and destination record.	Is the used asset the approved asset or a verified derivative?

The record should remain practical. It need not expose confidential instructions in every public file. It should let an authorised reviewer reconstruct the relevant chain and identify missing links. If the workflow cannot connect the approved candidate to the final rendition, the audit cannot assume they are equivalent because they look similar at thumbnail size.

Separate aesthetic quality from factual fidelity

Use different criteria for visual suitability and product accuracy. A pleasing composition can contain an invented handle or altered lid. Conversely, a faithful product image may need ordinary creative refinement. Combining both into one subjective score can conceal a material factual failure beneath strong aesthetic marks.

Compare the output with the approved source at the resolution and crop needed to inspect the relevant features. Include small details that affect the intended representation. If the catalogue uses several renditions, inspect whether resizing or cropping hides a qualification or changes the apparent product. The final user sees the delivered asset, not the full-resolution candidate in the review tool.

Where judgement is subjective, document the decision rule and reviewer competence. Two reviewers may reasonably prefer different backgrounds while agreeing that the capacity marking must not change. Preserve that distinction. A disagreement about taste should not be presented as evidence that factual acceptance criteria are unknowable.

Use a task-specific output review rubric

Original review rubric for generated product backgrounds

Dimension	Question	Evidence and disposition
Product identity	Are shape, lid and visible features faithful to the approved source?	Side-by-side source review; reject or correct material changes.
Claims implied by the scene	Does the image imply an unsupported accessory, capacity or use?	Product-owner adjudication; clarify or remove the implication.
Text and markings	Are Arabic and English markings preserved where required?	Qualified visual and language review, not OCR alone.
Creative suitability	Does the background meet the agreed brief?	Documented creative review separate from factual acceptance.
Provenance and permissions	Can required inputs, transformations and use decisions be traced?	Asset records and relevant permissions evidence; escalate unresolved questions.
Final rendition	Does the delivered file preserve the approved content?	Version comparison and destination evidence.

The rubric is an original internal method, not a universal legal or safety checklist. Criteria should be tailored to the product and intended audience. The audit should distinguish a proposed control design from evidence that reviewers actually applied it to selected operating assets.

Evaluate variability without selecting only the best examples

A workflow that generates multiple candidates needs an honest unit of evaluation. In a hypothetical exercise, twelve briefs produce four candidates each, or 48 images. If one image is selected per brief, twelve selected images and 36 unselected images do not automatically mean twelve successes and 36 failures. Unselected images may be acceptable but less preferred, or may contain material defects.

Define whether the claim concerns first-attempt quality, the probability of obtaining an acceptable candidate within a fixed process, or the quality of the final reviewed asset. Those questions require different denominators. Record the number of attempts, human edits and rejected defects needed to obtain the final asset, rather than describing curated outputs as representative raw generation.

Keep related candidates grouped when interpreting uncertainty. Four images from one brief share inputs and conditions; they are not four independent product scenarios. Use a justified mix of products, transformations and difficult features. The evaluation integrity guide addresses selection, automated judges and misleading benchmark claims.

Treat provenance as evidence with a specific meaning

The C2PA specification describes cryptographically verifiable provenance information and validation of its association and integrity. That is not a judgement that the depicted product is truthful or that every use is legally authorised. A valid provenance record and a misleading image can coexist.

Where the workflow uses content credentials or another provenance mechanism, test the actual chain through editing and delivery. Record what information is preserved, changed or lost, and how the final interface presents it. Do not assume a credential visible in the generation tool survives every export, crop, compression or publishing channel.

Absence of a credential is also limited evidence. It does not by itself prove that an image is synthetic, deceptive or unauthorised. The audit should examine the organisation's stated provenance requirement and what the observed workflow establishes. If a claim depends on a signed record, verify the relevant record and validation result rather than relying on a decorative badge.

Do not use synthetic-content detection as a universal truth test

A detector answers a particular classification question under its own conditions. It does not establish whether a catalogue image accurately represents the product, whether inputs were permitted or whether the correct version was approved. If a detector is used as a control, assess its task fit and evidence rather than assuming its label settles those other questions.

Test the detector on relevant benign fixtures and transformations within the authorised scope. Examine uncertain results and errors, and define the human decision that follows. Do not treat a low score as proof of human origin or a high score as proof of misconduct. The audit should preserve the distinction between a tool's output and a substantiated conclusion.

For many catalogue workflows, direct source-to-output fidelity review is more closely connected to the business question than asking whether the image looks AI-generated. That is a proposed audit judgement about this use case, not a claim that detection has no role elsewhere. Choose the control according to the consequence being managed.

Review Arabic text and local meaning in the asset

Arabic review must examine the rendered asset, not only the generation instructions. A model may produce plausible-looking letter shapes that change or obscure a marking. An OCR result can assist inspection but does not replace a qualified reader where the text matters. Preserve the approved wording and compare it with the final image at its actual display size.

Inspect the relationship between text and product. A capacity marking, a care instruction and a promotional phrase have different evidential roles. If a generated background introduces text that appears to be a product claim, the responsible owner should review that implication. Do not classify it as harmless decoration simply because it was not explicitly requested.

Where English and Arabic versions are prepared separately, verify that the same product facts, qualifications and intended use remain aligned. A translated caption does not correct a changed visual marking. Record language-specific findings and the final assets reviewed, rather than extending approval of one language version to every localisation.

Audit the human approval and publication boundary

Determine who can accept creative suitability, who can verify product accuracy and who can authorise publication. Those responsibilities may be held by different people. The reviewer should have access to the approved source and enough time to inspect material details. The human oversight guide examines whether those capabilities operate in practice.

Test a controlled version substitution: an approved candidate and a later edited derivative should remain distinguishable. If the derivative changes a material feature, establish whether renewed review is required under the policy. The audit should verify the actual destination file rather than rely only on an approval record in the creative tool.

Inspect handoffs to agencies, marketplaces or publishing teams where they are in scope. A correct asset may be cropped, relabelled or paired with an inappropriate caption downstream. Identify which part of the chain the engagement can examine. Supplier or channel restrictions should appear in the coverage record, not be hidden by a broad statement that the asset was audited.

Worked finding: the background edit changes the product

Suppose a controlled test generates a background around the approved jar photograph, but the selected output adds a feature to the lid. The creative review accepts the scene because the composition meets the brief. The source comparison shows that the feature is absent from the product photograph. This hypothetical finding concerns the failure to preserve a required product characteristic.

The observation does not establish that the generation model always changes lids or that a real catalogue was published inaccurately. It establishes the tested output and the review path that failed to detect the discrepancy. The cause might involve the editing mask, generation behaviour or review criteria. Preserve those as separate hypotheses until the evidence distinguishes them.

A suitable next decision may be to hold the asset, correct the product region and strengthen the factual review step. Retest the original transformation and relevant variations, then inspect the final rendition. A more attractive replacement is not proof of correction unless it preserves the required facts and follows the approval process.

Review changes to models, tools and output formats

A model update, editing tool, export format or localisation process can alter the workflow's failure modes. Record the relevant configuration and assess what earlier evidence remains applicable. A change that affects only background selection may need a different retest from one that redraws the product or changes text rendering.

Maintain known-defect cases and add fresh product scenarios. Repeatedly optimising the workflow on the same jar image can improve that example without establishing broader fidelity. Use retesting evidence to close the specific finding and preserve the limits of the new result.

Where the supplier controls the generation system, use the vendor-controlled audit strategy to identify observable evidence and restricted internal claims. Do not promise to verify all training inputs or reproduce every future asset when the available access cannot support those conclusions.

Adapt the method when the output is audio or video

The asset-chain approach can extend to other modalities, but the criteria need to change. An audio workflow may require review of spoken wording, speaker permissions and edited segments. A video workflow may require temporal consistency and inspection of frames that a thumbnail review would miss. An image audit does not automatically cover those additional questions.

Define the relevant unit and observation method for each modality. A short clip, one frame and a complete publication are different populations. Record transformations and the final delivered version, and use reviewers competent for the task. The audit should state which modality-specific procedures were actually performed instead of presenting a single generic content score as sufficient.

Examine reuse and correction after release

An asset may be reused in a later catalogue, resized for another channel or paired with a new caption. Identify whether the original approval covers that use and whether material product information has changed. A correct image of an earlier product version can become misleading when attached to a revised item. The audit should examine the asset-to-product relationship, not only the file's creation record.

Where a defect is discovered after use, inspect the process for locating affected renditions and replacing or withdrawing them through authorised owners. A corrected master file does not establish that every channel now uses it. Define the destinations the engagement can verify and preserve uncertainty about uncontrolled copies rather than claiming universal removal.

Keep the correction trail connected to the original finding. Record the affected asset, reason, approved replacement and evidence of the relevant handoffs. If the workflow allows an old file to be selected again, examine how that risk is handled. These are proposed operational audit questions; the appropriate response depends on the organisation's actual channels, authority and consequences.

Prepare a generative-system audit pack

Bring the intended transformation, approved inputs, generation settings, candidate records, edit history, acceptance criteria and one final asset with its approval trail. Identify which claims require product, language, security or legal expertise. The resulting review should connect the asset's creative purpose to the evidence supporting its use.

Discuss an audit of your generative AI content workflow to define the modalities, factual boundaries, provenance and approval controls that matter for the assets your organisation actually publishes or relies on.