AI Audit for Vendor-Controlled Systems

An AI audit for a vendor-controlled system must distinguish what the customer can observe, what the supplier can demonstrate and what neither party has made available for independent review. Restricted access does not make every audit impossible. It changes the questions that can be answered, the evidence needed and the limits that must accompany the conclusion.

This guide is for organisations using hosted AI products, managed platforms or embedded AI features whose internals they do not control. The hypothetical example is a vendor-hosted payroll enquiry assistant. It explains approved employer policies and directs employees to appropriate support. It does not calculate pay, alter payroll records or give an authoritative determination of an employee's entitlement. Tests use synthetic employees and approved policy fixtures.

NIST SP 800-161 addresses cybersecurity supply-chain risk, including the challenges created by reduced visibility into acquired technology. It is relevant background, not a guarantee about any vendor. The original approach below helps scope an AI audit around evidence that is both obtainable and capable of supporting the intended decision.

Map responsibility before requesting evidence

Separate the supplier's platform, the customer's configuration and the process shared between them. The vendor may control the model and infrastructure, while the employer controls policy documents, identity mappings, enabled features and employee communications. A finding at one layer should not automatically be attributed to another.

For the payroll enquiry assistant, identify who approves the policy source, who publishes updates, who grants access, who receives escalations and who can change model behaviour. Record the actual arrangement rather than assuming that a hosted product makes the supplier responsible for every outcome. The customer may create a material risk through its own configuration even when the underlying platform behaves as documented.

Also identify downstream dependencies. The assistant may call an identity service or create a support ticket through a separate provider. Testing the visible chat response does not establish that those operations completed correctly. The audit should name the boundary and obtain evidence from the party that can demonstrate the effect.

Turn broad assurances into answerable questions

“The vendor is secure” or “the AI is compliant” is too broad for a restricted-access assessment. Define the claim: employees see only their permitted information; answers use the current approved policy; escalation reaches the right support queue; or a specified customer setting is applied to the deployed tenant. Each needs a different evidence path.

Distinguish behaviour from process claims. Controlled input-output tests can examine how the assistant responds to selected questions. They cannot generally establish the full training-data history, every internal access path or complete deletion from all backups. Those claims need other evidence, and some may remain unresolved within the engagement.

Agree the decision the audit is meant to support. It may justify limited use with approved public policy information, while personal payroll queries require stronger access evidence. The conclusion should match that use boundary. A restricted audit can be useful without pretending to cover the entire supplier organisation or product portfolio.

Use an evidence ladder, without assuming more access is always better

The strongest evidence depends on the claim. Source-code access can help examine an enforcement mechanism but does not by itself establish which code operated in the customer's tenant. A witnessed test can demonstrate behaviour but may cover only a selected configuration. The audit should combine evidence where needed rather than treating one document or access level as universally sufficient.

Original evidence options for a vendor-controlled AI assessment

Evidence route	What it can support	What remains limited	Useful safeguard
Customer-side controlled tests	Observable behaviour for selected identities, inputs and conditions.	Internal mechanisms and unobserved processing.	Record tenant settings, timestamps and exact test fixtures.
Read-only configuration review	Settings visible for the named tenant or deployment.	Whether every setting operated throughout the period.	Connect configuration to operating evidence.
Supplier-witnessed tests	Behaviour and traces accessible through an agreed session.	Unwitnessed paths and independently reproducible access.	Agree cases in advance and include auditor-selected variations.
Restricted technical review	Specific implementation or control evidence under controlled access.	Deployment correspondence and broader historical operation.	Document scope, version and permitted retained evidence.
Third-party reports and attestations	The claims and periods actually addressed by those documents.	Customer-specific AI behaviour unless explicitly covered.	Check relevance, exceptions and complementary customer responsibilities.

The supplier evidence guide addresses authenticity, relevance and sufficiency of individual items. This page concerns how to build a workable audit strategy when access is constrained. A stack of unrelated certificates should not replace evidence about the customer's configured assistant.

Design observable tests with a reliable reference

For the hypothetical assistant, prepare an approved policy set with clear effective dates and synthetic employee contexts. Ask questions that the supplied material can answer, questions requiring clarification and questions reserved for payroll staff. Define the expected material behaviour before testing, including when the assistant should avoid an individual determination.

Test policy updates using the vendor-supported process. Record when the customer submitted the new source, when the product reported it available and which version supported the answer. A correct current answer does not establish that all earlier answers were current. Preserve the window and any uncertainty about propagation.

Use permitted identities to examine access separation and escalation. If the assistant has no personal-data access in the intended configuration, test that narrower role rather than inventing a payroll integration. If it does have access, the audit needs stronger identity and object evidence. Any security challenges must remain within agreed authorised boundary testing.

Make supplier-witnessed sessions evidentially useful

A demonstration selected entirely by the vendor can show a capability but may not challenge the relevant failure conditions. Agree the objective, environment, configuration and evidence beforehand. Include cases chosen by the auditor or customer within the authorised scope, and record which cases could not be executed.

Ask the supplier to show the relationship between the customer tenant and the demonstrated configuration. A special demonstration environment may use a different model, permissions or logging. Those differences do not make the session worthless, but they limit the inference. State whether the evidence establishes design feasibility, configuration correspondence or observed operation.

Preserve an agreed record of the session without exceeding access or confidentiality permissions. That may include case identifiers, configuration references, observed outputs and a supplier-confirmed trace extract. If the auditor cannot retain enough evidence for a reviewer to understand the conclusion, record that limitation and consider another evidence route.

Ask for targeted alternatives when direct access is refused

A refusal to disclose source code may be commercially understandable and irrelevant to a narrow output-quality question. It may be material to another question, such as how an access boundary is enforced. Respond by identifying the specific claim and the minimum alternative evidence capable of addressing it, rather than repeating a broad request for unrestricted access.

Alternatives can include a controlled technical review, a witnessed test with relevant traces, a scoped independent report or a configuration-specific explanation supported by operating records. Assess each alternative on its merits. A signed statement may establish what the supplier asserts, but it does not automatically provide the same evidence as an observed test.

Where no suitable alternative is available, preserve the unresolved question. The organisation can then decide whether to narrow use, apply a compensating control, seek contractual evidence rights or choose another product. Those are management and procurement decisions. The audit should not convert the vendor's refusal into either an automatic failure or an unsupported clean conclusion.

Use a claim-to-evidence gap record

Illustrative restricted-access plan for the payroll enquiry assistant

Claim	Available evidence	Gap	Decision or next evidence
Answers reflect the approved policy	Versioned source fixtures and controlled responses.	Internal retrieval trace unavailable.	Conclude on observed answers; qualify causal attribution.
Employee information is separated	Synthetic identity tests and tenant configuration.	Some internal access paths cannot be observed.	Seek targeted supplier evidence and state untested paths.
Escalation creates a support case	Chat request and receiving queue record.	None for the selected controlled cases.	Report the tested end-to-end outcome, not all historical transfers.
Customer content is deleted under the agreed policy	Deletion request and customer-visible disappearance.	Backend copies and backup lifecycle not established by that test.	Request relevant process evidence and preserve the limit.
The assessed configuration remains in use	Visible deployment identifier and supplier notices.	Underlying changes may not all be disclosed.	Define monitoring and reassessment triggers for the observable service.

The table separates a useful narrow conclusion from a missing broader one. A correct response can support the first claim for the tested case while leaving its internal retrieval mechanism uncertain. That uncertainty should not erase the observation, and the observation should not erase the uncertainty.

Do not overinterpret black-box results

A black-box test observes inputs and outputs without full internal access. It can demonstrate a failure, but often cannot identify the exact cause. If a payroll policy answer is wrong, possible causes include the customer's source, ingestion, retrieval, configuration or generation. The report should state what was observed and which causal explanations remain open.

Absence of a failure in selected tests is also bounded. It does not establish that all employees, policy variants, languages or hidden paths are protected. Define the cases and conditions, and avoid broad statements such as “the vendor cannot leak data”. The coverage guide provides statuses for conclusive, inconclusive and untested questions.

Do not infer deletion, absence of training use or complete data residency from an empty interface or a server location label. These are different claims requiring appropriate contractual, architectural and operating evidence. The relevant specialists should determine applicable obligations. A commercial technical audit should not turn a limited behavioural test into a legal compliance certificate.

Review customer-controlled settings as operating controls

Inspect who can change the assistant's sources, permissions, tools and retention-related settings. A supplier may provide a protective setting that the customer has not enabled, or the setting may apply only to a particular feature. Record the actual tenant configuration and the authority for changes.

Test the effect of a setting where the scope permits. A label in an administration screen is design evidence; a controlled observation can add evidence of behaviour. If the supplier documents dependencies or exceptions, include them in the test plan. Do not assume that toggling one setting governs every connected product or data flow.

Review customer communications and escalation ownership. Employees may believe the assistant is an authoritative payroll decision-maker even when its intended role is informational. That mismatch can create reliance risk without any vendor model defect. The audit should examine the configured presentation and actual process, not only the supplier's general documentation.

Handle vendor updates and limited reproducibility

Hosted services may change beneath a stable product name. Record all observable identifiers, settings and dates, and ask what change information the supplier makes available. If the underlying model cannot be pinned, describe the assessment as applying to the observable deployment during the test window rather than claiming permanent reproducibility.

Establish triggers for focused reassessment: a material feature change, new personal-data access, altered policy sources, an incident or a change in the vendor's relevant evidence. A routine interface update may not require a full audit, but it should receive an impact assessment when it affects the tested workflow.

Preserve regression cases for previously observed defects and add fresh cases when use changes. Connect ongoing evidence to continuous auditing where appropriate. A provider's notice of a fix is a useful input, but it does not by itself establish that the customer's configuration now behaves correctly.

Coordinate commercial rights without confusing them with evidence

An agreement may provide audit access, incident information or change notification rights. Those rights improve the ability to obtain evidence; they do not prove that a control operated. Record whether the rights were exercised and what the resulting review established. A clause promising cooperation is not the same as receiving the required trace or report.

The AI procurement guide addresses evidence-linked purchasing decisions. For an existing system, identify practical gaps in the current arrangement and the decisions they affect. Avoid requesting every possible document without a purpose, which can consume time while leaving the central question unanswered.

If the supplier will only permit a restricted review, assess whether the conditions allow a defensible conclusion. Confidentiality can be compatible with useful evidence, but limits on retention, reviewer access or disclosure may constrain what the customer can rely on. Make those consequences clear before describing the engagement as comprehensive.

Report responsibility, evidence and residual uncertainty

A useful report identifies the customer tenant, configuration, test window, supplier-controlled components and customer-controlled process. Findings should name the observed boundary and responsible owner where evidence supports that attribution. Where responsibility is disputed, preserve the competing explanations and the evidence needed to resolve them.

Separate verified operation, design evidence, supplier assertions and unresolved gaps. Management may accept a limited-use arrangement despite a gap, but that decision does not strengthen the underlying evidence. The report should state the accepted boundary and the conditions that would require more review.

When remediation is proposed, use retesting evidence to confirm the specific correction. A vendor ticket marked resolved or a newly issued policy statement is not enough for a behavioural finding if the affected path can be tested. Conversely, some process claims need documentary and period evidence rather than another chat demonstration.

Reconcile the supplier's evidence with the customer's experience

Compare identifiers and timing across the two sides of a controlled transaction. A supplier trace may describe a request that looks similar but belongs to another tenant, release or attempt. Agree how the records will be linked before the test. If a shared identifier is unavailable, document the alternative matching method and its ambiguity rather than presenting a guessed join as a complete trace.

Review failed and delayed transactions as well as successful ones. Customer logs may stop at the vendor request, while the supplier reports only completed processing. The missing cases can contain the most important uncertainty. Reconcile the intended test inventory and explain whether an absent result reflects failure, pending work, logging limits or a case that never reached the service.

For escalation, preserve the distinction between the assistant saying a case was created and the receiving queue accepting it. A supplier can demonstrate its outbound message while the customer's integration fails later. The responsible correction depends on the observed boundary. The audit should avoid forcing a single-party explanation when the evidence shows a shared process.

Use disagreement as a prompt for a focused evidence request. If the supplier says a setting was enabled but the customer observes different behaviour, preserve both observations and the configuration times. A later screenshot may not establish the earlier state. Ask what evidence would distinguish a configuration change from a product defect or a misunderstanding of the setting's scope.

Where the evidence cannot resolve the disagreement, state that explicitly and identify the decision affected. A temporary operating restriction may be reasonable while further work proceeds, but it is not a finding that either party acted improperly. Keep factual observations, competing explanations and the chosen response in separate parts of the record.

Finally, test whether the customer can continue the agreed controls after the auditor leaves. The supplier may provide a one-time evidence session without a recurring route for material changes. Identify which future decisions require renewed participation and what the customer can monitor independently. A defensible initial audit should not imply an ongoing evidence capability that the arrangement does not provide.

Prepare for a restricted-access audit

Collect the service agreement, configured feature list, responsibility map, policy sources, available tenant records and current supplier evidence. Identify what the organisation wants to rely on and what access it can authorise. A focused request makes it easier to negotiate useful evidence without implying rights the customer does not possess.

Discuss an audit strategy for your vendor-controlled AI system to determine which claims can be tested directly, which need supplier participation and which limitations must remain visible in the final decision.