AI Audit under NCA ECC

An AI audit under NCA ECC should show whether the cybersecurity controls relevant to a defined AI service have been implemented and are operating. It should identify the legal entity, the applicable controls, the technology boundary and the evidence supporting each conclusion. A list of model risks cannot do that job on its own. Nor can a supplier certificate explain which customer configuration, connector permissions or administrative activities were examined.

This guide is for Saudi cybersecurity leaders, internal audit teams and technology owners preparing an ECC-related review of an AI system. It uses a hypothetical internal IT support knowledge assistant to show how to turn a broad control requirement into a defensible test. The proposed procedures are original audit design examples; they are not an NCA-issued AI checklist or a substitute for the entity's applicability assessment.

Establish which entity and requirements are in scope

The NCA Essential Cybersecurity Controls, ECC-2:2024, cover Saudi government bodies and affiliated entities, and private organisations owning, operating or hosting critical national infrastructure. Other organisations are encouraged to use them. Subdomain 1-8 addresses cybersecurity review and independent audit. These provisions do not make every private business subject to identical obligations.

Begin with a short applicability memorandum approved by the responsible cybersecurity and compliance functions. Record the entity's status, the services and assets included, the source and version of each requirement, and the reason for every exclusion. If the AI service crosses subsidiaries, do not assume the parent's conclusion automatically describes every subsidiary. Equally, outsourcing a component should not remove it from the system map simply because its operator is elsewhere.

Separate three questions. First, is the entity subject to the requirement? Second, does a particular control apply to this technology and service? Third, has the control been implemented effectively? An answer to the first question is not evidence for the third. A cloud service that has not yet launched may still need review before use; an inactive experimental connector should be checked for reachable accounts rather than dismissed based on its label.

Ask the accountable owner to resolve uncertainty before a compliance percentage is calculated. Marking an unclear item as not applicable artificially improves the result. An unresolved applicability question should have an owner, a due date and a statement of how it limits the audit conclusion. This also prevents a technical assessor from silently making a legal interpretation outside the agreed engagement.

Draw the security boundary around the complete AI service

In the illustrative assistant, employees ask questions about approved IT procedures. The application retrieves knowledge articles, sends selected passages to a model and returns an answer. Its boundary includes the employee identity, application session, retrieval index, source repository, model endpoint, logging pipeline, administrative console and supplier support route. The assistant has no authority to reset passwords or change access rights. That restriction is an explicit audit criterion, not an assumption based on its current interface.

A useful boundary diagram distinguishes data movement from authority. A search connector may read many documents while the user is authorised to see only a subset. A model may produce a suggested command while a separate service holds execution permission. A monitoring platform may receive content even though the business application stores only an interaction identifier. Reviewers need each distinction to select the right enforcement point and avoid testing an irrelevant layer.

Record the actual deployed versions and configuration snapshots. A clean demonstration using a new index does not establish what happened when last month's index still held a restricted article. Include the account used for each connector, the environment, enabled tools, relevant network routes and administrator groups. Where infrastructure is shared, identify which controls are inherited and which remain the responsibility of the application team.

Use an asset owner interview to discover boundaries, then corroborate the interview with read-only configuration and service records. Interviews can explain intent but cannot establish that a permission was absent. For a supplier-operated element, record the specific evidence available and the evidence unavailable. The AI auditing service should be scoped around those access conditions before anyone promises a full technical conclusion.

Convert control themes into evidence and tests

The following matrix is a proposed way to examine this assistant. It connects familiar cybersecurity themes with AI-specific assets without presenting a new regulatory control set. The entity should maintain the authoritative mapping to its selected ECC provisions in a separate workpaper, including the exact requirement and the justification for applicability.

Illustrative control-to-test map for an IT knowledge assistant

Control theme	Evidence to request	Proposed audit procedure	Limit on the conclusion
Identity and access	Role definitions, connector identity and current repository permissions	Compare permitted retrieval for an ordinary user and a privileged test user using synthetic articles	A blocked answer alone does not show that restricted text never reached the model
Data protection	Content-flow inventory, redaction settings and logging configuration	Follow a harmless marked test value through the permitted processing path	An absent value in one log does not establish absence from every supplier system
Change control	Approved release record, index build identity and deployment evidence	Reconcile selected live configurations to authorised changes	A ticket marked complete does not establish which version was running
Monitoring and response	Alert rules, event samples and incident ownership	Witness an authorised test alert and trace its receipt and disposition	A generated event is different from an alert received by the responsible team
Third-party arrangements	Service scope, support permissions and evidence-access terms	Check whether supplied assurance covers the purchased service and relevant period	Supplier-wide assurance cannot fill every customer configuration gap

Each row should become one or more workpapers with a named criterion, population, test steps and expected result. Avoid combining several controls into a single pass box. An assistant can enforce employee authentication and still expose documents belonging to another department. A release can be authorised while its monitoring rule is disabled. Separate results preserve the information needed for remediation.

Worked test: does a permission change reach retrieval?

Suppose a synthetic article describes a fictional support procedure and contains a harmless marker. The article initially belongs to a general IT group. An authorised administrator then moves it into a restricted test collection. The audit question is whether the assistant respects the changed permission throughout the retrieval path within the organisation's own approved synchronisation rule. The test does not use real confidential documents or undisclosed user accounts.

1. Record the source permission, user role, index version and approved synchronisation expectation before the change.
2. Confirm that the ordinary test user can retrieve the synthetic article while it is intentionally accessible.
3. Apply the authorised permission change and record when the source system confirms it.
4. Repeat permitted queries before and after the expected update point, recording retrieval results separately from generated answers.
5. Inspect the authorised application evidence to determine whether restricted text entered the model context, appeared in a response or was retained in an audit-visible log.
6. Repeat with a fresh session and a previously active session, then document any cache-related difference.

The result may be more subtle than a visible disclosure. The answer can refuse to reveal the procedure even while the restricted passage is present in model context. That is evidence about the response control; it is not evidence that the retrieval boundary worked. Conversely, an empty retrieval result caused by a broken test connector does not demonstrate access enforcement. The positive control in step two helps distinguish those explanations.

Use the entity's approved rule to judge timing. Do not invent a universal NCA synchronisation deadline. If the organisation has no defined maximum delay or compensating restriction, record that design gap and its consequences. If a policy requires immediate revocation but the architecture updates permissions periodically, the disagreement must be resolved explicitly. Changing the wording after observing failure should not retroactively change the original result.

This procedure is a bounded example of AI security boundary testing. A wider engagement may add authorised checks for administrative access, network restrictions or tool permissions. Each extension needs its own objective, safe test conditions and stop rule; the initial retrieval test does not authorise unrestricted penetration testing.

Test operation over time, not only today's configuration

A current screenshot can establish what a setting displayed at the time of capture. It cannot establish that the setting remained active throughout the review period. For operating effectiveness, obtain a relevant population such as releases, account changes, access reviews or security events. Reconcile the population to an independent system record where possible, and explain any period for which reliable history is missing.

For example, a hypothetical review identifies 18 production releases during the agreed quarter. The assessor selects six based on defined risk characteristics, including a connector change and a model endpoint change. If two selected releases lack evidence of the required review, report two exceptions among six selected releases. Because selection was targeted, do not describe that fraction as the rate across all releases or across all organisations.

Investigate whether missing evidence reflects an unperformed control, inaccessible records or a different approved evidence location. These explanations can lead to different findings. A verbal statement that an engineer remembers approving the change may guide further enquiry, but it does not automatically repair the missing record. If independent corroboration is unavailable, retain the limitation rather than converting uncertainty into a pass.

For an AI service that changed significantly mid-period, split the conclusion by version or period. It can be appropriate to report that the old retrieval configuration failed and the corrected configuration passed specified retests. Merging both into a single green status hides the historical exposure and overstates the duration for which the new control was observed.

Keep independence and evidence handling explicit

Identify who designed the control, who operates it, who selects the sample and who signs the conclusion. A reviewer who helped build the system may contribute technical knowledge, but the resulting self-review threat needs an engagement-specific response. Depending on the purpose, that can mean separating readiness work from independent testing, changing reviewers or obtaining a separate provider. See AI audit independence and conflicts of interest for the decision process.

Agree access conditions before collecting evidence. Read-only access, witnessed sessions, restricted exports and masked samples can reduce unnecessary exposure, but each can also limit reproducibility. Record which method was used and who controlled the evidence selection. A supplier-selected demonstration should be labelled as such; it should not be described as an assessor-selected production sample.

Audit records can contain personal information, architecture details and security-sensitive configuration. Establish an authorised repository, access owners, transfer method and retention decision. Preserve the minimum evidence necessary to substantiate the finding while keeping enough context for review. AI audit evidence handling and PDPL-focused audit work address different parts of that responsibility; compliance with one does not silently resolve the other.

Reconcile the inventory before relying on inherited controls

An inventory gap can undermine an otherwise careful test programme. In the example, the technology owner supplies an approved diagram with one model endpoint. A read-only review of permitted deployment records identifies a second endpoint used during fallback. The auditor should first establish whether that route can actually process the scoped service's requests. An old configuration string is a lead to investigate, not proof of current data transfer. Conversely, a route excluded from the diagram should not be ignored if current execution evidence shows it was used.

Reconcile three views: the service owner's declared components, the configured components visible within authorised access, and the components evidenced in selected activity records. Explain disagreements by reference to versions, environments and time. A development-only endpoint may sit outside a production review, but only after its separation is supported. A shared gateway may serve several applications; obtain enough attribution to avoid treating another application's activity as evidence about this assistant.

Next, build an inheritance record for each shared control. Identify the control provider, the specific service covered, the evidence date and the customer action needed for the control to work. A central identity platform can authenticate a person while the application remains responsible for deciding which documents that person may retrieve. A platform backup can preserve infrastructure without preserving the source permissions needed to reconstruct a defensible historical test. The inherited and local responsibilities should meet at a documented boundary.

When a shared control changes, determine which conclusions need reconsideration. Replacing an identity provider may affect authorisation assumptions across several connectors; changing a cosmetic interface label usually will not. Use a documented impact decision rather than automatically repeating every test or leaving the old report permanently valid. The review should identify the old evidence, the changed dependency and the additional work necessary to support a current conclusion.

Finally, reconcile totals without confusing them with assurance. Twelve listed components and twelve evidence folders demonstrate inventory coverage only if the folders contain the relevant evidence. They do not establish twelve effective controls. Report the inventory's completeness assessment separately from control outcomes, so management can see whether a gap concerns an unknown asset, a missing record or an observed failure.

Write findings that lead to a verifiable corrective action

An illustrative finding could state: “For the tested assistant configuration, a permission change in the source repository did not remove a synthetic restricted article from model context after the organisation's approved update window. The application response withheld the article text, but retrieval evidence showed that the source access boundary was not enforced as specified.” This describes what was observed without claiming an actual confidential-data breach.

The finding should reference the criterion, test identity, relevant timestamps and configuration. Record the consequence as a reasoned exposure: restricted source content could be provided to a downstream processor despite the source restriction. Keep the cause provisional until investigated. A stale index, a shared connector account and an incorrectly cached user decision can create similar symptoms but require different corrections.

A useful remediation plan names the enforcement change, the owner and the evidence required to demonstrate effectiveness. It should also address the period before correction, including whether the affected feature needs restriction. Closing the ticket after code deployment is insufficient when the original issue concerned operating behaviour. AI audit retesting should repeat the failed case, add relevant variations and confirm that legitimate access still works.

Management may accept a residual risk within its authority, but that decision should remain distinguishable from a passed test. It cannot convert an applicable legal requirement into an optional one. The report should preserve open limitations, the authority for any acceptance and the planned review trigger, rather than treating all administratively closed items as technically resolved.

Build a report that can withstand follow-up questions

The reporting pack should let a reviewer move from the applicability decision to the tested system, selected evidence, findings and conclusion without relying on the assessor's memory. Include the criteria register, scope statement, evidence index, sample rationale, findings and remediation status. Provide a short executive explanation of what the audit changes about the organisation's understanding of exposure, alongside the technical detail needed by control owners.

State exclusions prominently. If supplier infrastructure, historical model context or a production connector was inaccessible, identify the affected questions. Do not bury a limitation in an appendix while describing the entire AI platform as compliant on the cover. Equally, avoid turning an isolated, well-bounded exception into a claim that the complete organisation failed every cybersecurity obligation.

Before commissioning work, prepare the system inventory, applicability memorandum, recent changes, supplier boundaries and a masked example of the evidence available. Scoping an AI audit helps turn those inputs into a feasible engagement. Request an audit scoping discussion for an ECC-related AI review to agree the control objectives, authorised testing and report audience. The outcome sought is a supported conclusion with actionable limits, not a generic compliance badge.