AI Audit for Government

An AI audit for a Saudi government service should examine how the system affects the public service journey, the authority to make decisions and the evidence available when a result is challenged. A helpful-looking answer can still misstate a requirement, send an applicant into the wrong route or make a preliminary machine assessment appear to be an official decision.

This guide is for government service owners, internal audit, digital teams, cybersecurity and data-governance functions. It uses a hypothetical assistant that checks the apparent completeness of a permit application and drafts an internal note for an authorised caseworker. The assistant does not grant or refuse the permit. The audit examines that distinction, the handling of changing requirements and the applicant's path to a responsible human decision.

Define the public-service purpose and the limits on authority

SDAIA's Generative Artificial Intelligence Guidelines for Government address responsible government use, human review and restrictions concerning critical decisions affecting individuals or the Kingdom's vital interests. Treat the document as official guidance with its own scope, alongside the entity's binding requirements and policies. Do not turn it into an invented rule covering every type of AI identically.

For the illustrative permit service, write the intended function narrowly: identify apparent missing items against an approved requirements source and prepare a review note. List excluded actions, such as issuing a refusal, changing a statutory condition, deciding eligibility or making an official commitment to an applicant. The actual interface and integrations should support those boundaries.

Identify the authorised decision-maker and the process that gives a decision official status. A caseworker's review, a committee determination and an administrative publication can be different events. The audit should not assume that a model-generated note becomes valid merely because it is stored in a government system. Map the point where the note can influence the official record.

Scope the service journey from submission to resolution

Follow an application through submission, document receipt, AI analysis, caseworker review, applicant communication, correction and final disposition. Include assisted channels where relevant, such as a service centre or an authorised representative. A portal-only test can miss the different information available to staff handling the same case through another channel.

Record the legal entity, service, application type, languages, versions and review period. Identify systems outside the audit, including external registries or downstream notification platforms. Where their behaviour matters to a conclusion, request suitable evidence or retain a limitation. Do not describe the whole public service as audited when only one assistant interface was examined.

Ask what happens to an applicant when the AI is uncertain or unavailable. Can the case still reach an authorised reviewer? Is the reason for delay visible? Does the applicant receive an instruction that can be acted on? These are proposed audit questions to tailor to the service, not a substitute for the entity's approved service standards or legal duties.

Use an authoritative requirements source with version evidence

The assistant should be evaluated against the requirements actually approved for the service and relevant application category. Identify the authority for each source, its effective date and its relationship to public-facing instructions. A search result, archived notice or supplier-authored summary should not silently become the controlling rule merely because it is easy for the model to retrieve.

Compare Arabic and English versions where both are provided. If their meaning differs, the responsible service and legal owners should determine the authoritative interpretation. The audit should record the inconsistency and how it is handled rather than let the model choose whichever version is more convenient. Preserve the source version used for each examined result.

Test the transition when a requirement changes. An application submitted before the change may need different treatment from a new application, according to the service's approved rules. The auditor should obtain that transition decision rather than assume either universal retrospectivity or a permanent exemption for old cases. The technology must follow the authorised policy choice.

Original public-service evidence map for an application-completeness assistant

Decision boundary	Evidence to request	Proposed audit question
Applicable requirement	Approved source, effective date and application-category rule	Did the assistant use the correct version for this case?
Preliminary assessment	Input record, generated note and uncertainty handling	Is a suggestion clearly separated from an official determination?
Human decision	Authority, source access, rationale and recorded disposition	Can the caseworker correct the note before it affects the applicant?
Applicant communication	Approved wording, channel and delivery evidence	Does the message state a supported requirement and a usable next step?
Correction and challenge	Case history, revised documents and review route	Can an error be corrected without losing the original decision trail?

Worked test: an outdated requirement enters an official-looking message

Assume a hypothetical permit service has replaced one supporting-document requirement. The approved transition rule applies the new list to applications received from a specified effective date. A synthetic application submitted after that date is complete under the new list. The assistant's retrieval source still contains the old requirement and drafts a message asking for an unnecessary document.

1. Record the approved old and new lists, effective date and transition decision.
2. Create a fictional application with the documents required under the new rule.
3. Run the assistant in the authorised configuration and capture the source references and draft note.
4. Trace the note to the caseworker interface and the applicant-message workflow.
5. Check whether an authorised reviewer can identify the outdated requirement, correct it and prevent the unsupported request from being sent.
6. Verify that the corrected outcome and the reason for correction remain associated with the case.

The test should distinguish retrieval failure from decision-control failure. The assistant may retrieve an outdated source while a reviewer catches the problem before communication. That supports a finding about the source process and evidence of a compensating review in the tested case. It does not justify claiming that the retrieval issue is harmless across all cases.

If the message is automatically sent, determine what authority and process allowed that action. Do not infer that every automated communication is prohibited; the question is whether this particular message and function meet the approved service rules and applicable guidance. The report should state the observed path and the decision boundary it crossed.

Use RAG system auditing for a deeper examination of retrieval and source lineage, but keep the government-service review focused on the public consequence and responsible decision. A correct citation to an obsolete document can still support the wrong instruction for the current application.

Test uncertainty and incomplete inputs without creating a hidden refusal

An incomplete scan, unfamiliar document or ambiguous application category should have a defined handling route. The assistant may be unable to assess completeness, which is different from concluding that the applicant failed a requirement. Inspect the labels, queue rules and messages that carry that distinction through the service.

Use authorised synthetic cases with missing pages, unreadable sections and uncertain categories. Check whether the system asks for a justified clarification, refers the case to a responsible person or produces an unsupported negative conclusion. The expected outcome should follow the entity's approved process. Do not invent a national service rule to make the test easier.

Examine queue behaviour. A case labelled cannot assess may remain indefinitely in a technical exception queue unless someone owns it. Measure its age using a defined starting point and account for paused or reassigned cases. An applicant should not disappear from the audit population simply because the AI did not return a normal result.

Assess Arabic communication for administrative meaning

Review the difference between a request, a recommendation, a preliminary observation and an official decision in the wording shown to users. Arabic phrasing should make the status clear without unnecessary complexity. A polished sentence can still imply that the authority has decided something when the system has only identified a possible missing item.

Test dates, document names, conditions and exceptions against the approved source. Where English material is also used, verify that translation preserves the conditions rather than only the broad topic. The reviewer should be able to locate the exact source supporting the instruction. Generic language quality scoring is insufficient for a message that changes what an applicant believes they must do.

Include practical accessibility of the explanation in the scoped review. An instruction may be accurate but fail to identify which document or page needs attention. A message may use an internal code that applicants cannot understand. The audit can test whether the communication provides a usable next step while preserving the entity's authorised wording and channel requirements.

Keep authority, access and data classification connected

Government AI services can involve information with different access and handling requirements. The entity's data-governance and cybersecurity owners should identify the applicable classification and controls. NCA's Essential Cybersecurity Controls provide an important applicable framework for government entities; the audit must still map the relevant service controls and evidence.

Test whether the assistant can access only the records and sources appropriate to the caseworker and task. A central connector may have broader access than an individual employee. The audit should examine enforcement at the actual retrieval or service boundary, using authorised synthetic records. A prompt asking the model to keep information confidential is not equivalent to a permission control.

Review external processing and support routes through the responsible approval process. Identify what content leaves the entity, which party receives it and what evidence supports the arrangement. PDPL-focused auditing may be relevant to personal-data processing, while NCA-related review examines selected cybersecurity controls. Keep their criteria explicit rather than treating one successful test as complete coverage.

Evaluate performance in terms of service outcomes

Define what the assistant is expected to improve and what errors matter. Useful categories can include unsupported document requests, missed required items, unresolved uncertainty, incorrect category selection and messages requiring correction. The definitions should be agreed before evaluation and tied to the authorised task, not chosen afterwards to make the result look strong.

Separate model output quality from overall service time. A fast draft can still create additional work if caseworkers must correct it or applicants submit unnecessary material. If the audit examines efficiency, include review effort, repeat contacts and unresolved cases in the agreed measure. Do not claim time savings from model response speed alone.

Use a documented reference process with people competent in the service rules and language. Record ambiguous cases and disagreements. Evaluate ordinary cases and targeted difficult cases separately, explaining the selection mechanism. Audit sampling helps avoid treating a deliberately challenging set as a population error rate.

Test correction, challenge and the history of a case

A public-service record should allow an authorised reviewer to understand how the relevant result was reached and changed. For the scoped audit, trace the source version, AI note, human disposition, applicant message and subsequent correction. The record need not expose unnecessary technical detail to the applicant, but it should support the entity's accountable review process.

Use a fictional case in which an applicant supplies a corrected document or challenges an apparent missing-item request through the approved route. Examine whether the new material reaches the responsible reviewer, whether the AI's previous note remains distinguishable from the revised assessment and whether the final communication reflects the authorised decision.

Do not overwrite history in the name of presenting a clean record. A corrected outcome should preserve the relationship to the earlier error according to the entity's approved recordkeeping rules. Conversely, retaining every raw model interaction without a purpose can create unnecessary data exposure. Audit evidence handling helps distinguish necessary review evidence from indiscriminate collection.

Assess supplier deliverables against the public service boundary

A supplier demonstration may show fluent answers while omitting the official-message route, caseworker permissions or requirement updates. Ask for evidence relevant to the purchased service and the entity's configuration. The audit should identify which responsibilities belong to the supplier, the implementing team and the government service owner.

Inspect acceptance criteria before relying on a completed project status. Did acceptance test the correct requirements version, uncertainty handling, Arabic administrative meaning and correction path? Were unresolved issues documented and assigned? A project sign-off may establish contractual acceptance without establishing every operating control needed for continued public use.

For changes after acceptance, determine how the entity learns about model, retrieval and interface updates. The service owner should know which claims need fresh evidence. A supplier's generic release announcement may not explain the effect on a particular public-service instruction, so the audit should examine the local impact assessment rather than stop at the existence of an email.

Report findings without implying an official approval

An illustrative finding could state that a synthetic post-change application received a draft request based on an obsolete requirement, and that the examined message route did not require the specified review before sending. Identify the configuration, source and authorisation evidence. Do not claim that all applicants were affected or that a particular legal violation is established without the relevant evidence and interpretation.

Define corrective actions by mechanism: update the source publication process, fix version selection, require the appropriate review or improve exception ownership. Retesting should verify the original failure and relevant before-and-after-date cases. A new prompt alone may not resolve an obsolete source or an over-permissive integration.

Present the public-service consequence and the remaining uncertainty to the authorised management audience. State which functions were tested and which need further work. An audit report from a commercial provider is not a government endorsement, procurement approval or permission to expand an AI function beyond the agreed use.

Reconcile channel coverage and effective dates

A service-level conclusion needs a service-level population. Suppose a hypothetical review period contains 120 applications: 95 submitted through the portal and 25 through an assisted channel. If the AI evidence export contains only the portal cases, the auditor should not describe it as the complete service population. Determine whether the assisted cases use the same assistant, a different process or no AI at all, then define the coverage accordingly.

This is not a reason to force every channel into the same technology. It is a reason to explain who is included in the evidence and how the service handles people outside that route. If assisted applications enter later through a staff interface, trace their original submission context and identify which rules apply. A technical upload timestamp may differ from the service's recognised receipt date.

Effective-date tests should use the entity's approved date and time conventions. A display such as 01/02 can be ambiguous without a defined format, and a converted calendar value should not silently replace the source record. Where calendars or time zones differ between systems, examine the documented conversion and preserve the original event evidence. The audit does not need to invent a universal conversion rule; it needs to verify the rule actually authorised for the service.

Test a controlled case near the transition boundary and inspect the selected requirement version. If the result differs between the portal and staff interface, investigate the source of the difference before assuming either route is correct. The finding should identify whether the issue concerns date interpretation, channel mapping or an unresolved policy decision. These distinctions help the service owner correct the responsible mechanism and explain the affected population.

Prepare the service owner for a focused audit

Assemble the service map, approved requirements and transition rules, authority model, message templates, data-handling decisions, supplier responsibilities and a masked case history. Include staff responsible for applicant experience as well as technology and governance. They can explain where a technically plausible output becomes a confusing or consequential instruction.

Trusted's government and public-sector page provides commercial context on possible AI application areas. Use that as exploratory reading, while relying on the entity's actual rules and authoritative sources for the audit criteria and any compliance conclusion.

Request an audit scoping discussion for a government AI service through OpenForAI's AI auditing service. The engagement should help the entity understand the evidence behind its service decisions, correct specific weaknesses and preserve clear accountability to the people using the service.