AI Audit for Healthcare

An AI audit for Saudi healthcare should examine the clinical or operational decision that an AI output can influence, the evidence supporting that use and the controls around the complete workflow. A model benchmark alone cannot establish that the right patient context reached the application, that a clinician reviewed the correct draft or that a later amendment reached the people relying on the report.

This guide is for hospital governance teams, clinical informatics leaders, quality functions and technology owners commissioning a scoped review. Its hypothetical example is a radiology report-drafting assistant. The audit examines draft handling, source fidelity, clinician review and amendment propagation. It does not diagnose patients, determine treatment or replace clinical validation, medical-device assessment or the organisation's authorised clinical decisions.

Classify the intended use before selecting audit tests

The Saudi Food and Drug Authority's guidance for AI and machine-learning-enabled medical devices links regulatory treatment to intended use. A hospital should have the responsible specialists assess the actual function and claims. Not every administrative AI tool is a medical device, and describing a clinically influential function as documentation support does not settle its classification.

Write down what the assistant receives, produces and influences. Does it restructure clinician-supplied observations, add inferred findings, suggest interpretations or generate a final report? Which functions are enabled in the purchased configuration? Compare the product instructions, procurement description and actual interface. A broad product family description may not accurately describe the specific function being deployed.

Record the applicable approvals, restrictions and specialist assessments without treating the audit as their substitute. If classification is unresolved, identify who must resolve it and which use decisions remain constrained. The audit can supply factual evidence about system behaviour while preserving the distinction between technical observation and a regulatory or clinical conclusion.

Define the clinical workflow boundary and decision authority

In the illustrative workflow, an authorised clinician provides source material for a draft, reviews the generated text and signs the final report through the approved reporting system. The assistant is not authorised to sign, distribute a final clinical conclusion independently or silently overwrite an approved report. Those boundaries should be reflected in permissions and workflow state, not just a message on the screen.

Map the encounter and document identifiers through the source system, drafting application and report destination. Identify where a user selects context, where the draft is saved and where finalisation occurs. A technically correct draft attached to the wrong encounter is a different failure from an inaccurate statement within the correct draft. Both require evidence beyond a language-quality score.

Include downstream use within the agreed scope. A report may be viewed in a clinical system, exported, printed or sent through an interface. A later amendment may not reach each route in the same way. The audit should identify which destinations are examined and which remain outside coverage, so the conclusion does not imply a complete end-to-end review when only the drafting interface was tested.

Separate source fidelity from clinical correctness

Source fidelity asks whether the draft preserves the information supplied to it, including negation, uncertainty, location and relevant qualifiers. Clinical correctness asks whether the underlying information and interpretation are clinically appropriate. The first can be tested with controlled source-and-output comparisons; the second requires suitable clinical expertise and evidence for the intended use. Neither should be silently substituted for the other.

For example, a synthetic source can contain an explicit uncertainty statement that the draft turns into a definite statement. The audit can identify that transformation without claiming to determine a patient's condition. Another draft may preserve the source accurately while the source itself is incomplete. Record these categories separately so remediation addresses the correct part of the workflow.

Use a predefined error taxonomy developed with qualified reviewers. Possible audit categories include omitted source content, unsupported additions, altered negation, changed laterality, incorrect document association and loss of an amendment. These are proposed evaluation categories, not clinical thresholds or a claim that any particular system produces those errors. Define the consequence and escalation process for each relevant category.

Original evidence map for a healthcare report-drafting audit

Claim	Evidence to examine	Test focus	Conclusion boundary
Correct encounter context	Identifiers, selection controls and interface records	Controlled context-switch and association cases	Does not validate the clinical content itself
Faithful draft	Authorised source, draft version and reviewer annotations	Preservation of specified facts and uncertainty	Does not prove every possible input is handled correctly
Meaningful clinician review	Review interface, authority and disposition history	Ability to inspect, reject and correct before sign-off	A supervised exercise is not full historical operating evidence
Controlled finalisation	Report state, permissions and destination evidence	Separation of draft from authorised final report	Applies to the tested routes and configuration
Amendment propagation	Version links and downstream acknowledgement	Visibility of a correction in selected destinations	Untested external recipients remain a limitation

Worked test: does the final record preserve an amendment?

Use a fictional encounter and synthetic report text in an authorised environment. The test begins with a clinician-approved draft, then introduces a harmless controlled amendment through the normal reporting process. The question is whether the corrected version becomes visible in the selected destination and whether the previous version remains identifiable as superseded where the approved process requires that distinction.

1. Record the encounter identifier, source version, draft version and application configuration.
2. Complete the authorised review and finalisation route using the synthetic case.
3. Confirm which version appears in each scoped downstream view or export.
4. Apply the controlled amendment through the approved user role and preserve its reason and timestamp.
5. Inspect downstream evidence for the corrected content, version relationship and any acknowledgement available.
6. Test a delayed interface or retry condition safely, and record whether the final state can be reconciled.

A successful send event does not necessarily establish that the destination displayed the amendment. Similarly, a timeout does not prove the destination rejected it. Reconcile the final record using authorised destination evidence. Where that evidence is unavailable, report the unverified delivery state rather than mark the correction complete because the source system queued a message.

If the amendment arrives but the interface still displays an older cached version, distinguish storage from presentation. If the wrong encounter receives it, treat association as a separate issue. If the system creates two apparent final versions, investigate ordering and state handling. The original test should preserve enough detail to distinguish these mechanisms without collecting unnecessary patient information.

The example is a workflow test, not evidence of clinical safety for a product. A real deployment may require additional clinical, regulatory and technical validation. Its value for the audit is that it follows a consequential correction beyond the model output and identifies exactly which claim the available evidence supports.

Choose a review population that reflects the intended use

Define the population before selecting cases: report types, languages, source conditions, users, sites, versions and time period. Identify exclusions such as unsupported input formats or unavailable historical source material. A test set selected for clean formatting can miss the conditions that challenge the actual drafting workflow. Conversely, a deliberately difficult challenge set should not be described as a representative error-rate sample.

Use qualified reviewers and a documented reference process. Reviewers should know the task and the permitted source material, and disagreements should be recorded and resolved through an agreed method. If a model is used to assist scoring, examine the validity and limitations of that scoring method. The system under review should not be treated as the unquestioned judge of its own correctness.

Report error categories and denominators. In a hypothetical set of 80 synthetic report drafts, three may contain an unsupported addition and two may contain a changed qualifier, with one draft appearing in both categories. That means four distinct affected drafts, not five. Preserve overlap and avoid converting this purposeful test into a clinical population estimate. Evaluation integrity helps structure these decisions.

Examine bilingual performance at the level of meaning

Arabic and English review should examine whether the relevant meaning survives, not merely whether the text is fluent. Record the language of the source, the requested output and any intermediate translation or transcription. Mixed-language terminology, abbreviations and numerals should be included where they occur in the intended workflow, with suitable reviewers for each condition.

For a synthetic case, test whether uncertainty remains uncertainty and whether a negated statement remains negated after drafting or translation. Preserve document and version identifiers exactly where required by the application. Avoid assuming that an English evaluation automatically supports an Arabic output configuration. The report should identify supported language conditions and those that were not evaluated.

Assess the interface as well as the text. A reviewer may need to compare the source and draft while using a bilingual system. If an important qualifier is hidden in a collapsed field or truncated export, a language-correct model output may still be difficult to review. This is a workflow and usability question requiring its own evidence.

Audit human oversight as an observable control

Identify the person authorised to review and the decision they can make. Check whether they can access the necessary source, see that the text is a draft, edit or reject it and prevent unapproved finalisation. A sign-off field is not enough if the system publishes the draft before the review or if another route bypasses the reviewer.

Use controlled cases to observe the correction process, then examine selected operating records if the scope includes ongoing effectiveness. Consider workload and exception handling without assuming that every quick review is inadequate. The key evidence is whether the reviewer had the required information and authority and whether their decision affected the final record. Human oversight auditing provides the dedicated control method.

Distinguish training from demonstrated capability. Attendance at an AI awareness session does not show that a clinician or operator can identify the application-specific failure being tested. A practical exercise can evaluate that capability, but should be designed with the responsible clinical and operational owners and should not interfere with live care.

Protect health information in the audit itself

The PDPL Implementing Regulation includes health-data protections in Article 26. The organisation's privacy and healthcare specialists should determine the applicable processing and evidence-handling conditions. The audit should request only the information needed for its authorised objectives and should not create uncontrolled copies of health records.

Prefer synthetic fixtures for technical boundary tests where they can answer the question. Where real records are necessary for an approved evaluation, define access, minimisation, masking, secure review and retention. Removing a name may not remove every identifying detail from a narrative. Record the method and its limitations rather than describe all masked records as fully anonymous.

Review secondary copies in screenshots, support tickets, exports and workpapers. A strong clinical application access model does not protect an audit folder shared too broadly. Evidence handling and privacy-focused AI auditing should be coordinated with the healthcare review, with clear responsibilities for any processing decisions.

Assess suppliers and changes without relying on broad claims

Request the supplier's intended-use statement, supported configurations, relevant authorisation information, evaluation evidence, change notices and access conditions. Match them to the exact purchased service and version. A publication about a research model or a demonstration of another product variant does not automatically establish performance of the hospital's configured workflow.

Identify changes that may affect the evidence: a model update, new input format, altered prompt, revised reporting template or another interface destination. Determine whether the organisation assesses those changes before relying on old results. A minor wording adjustment and a change that affects report meaning should not be treated as equivalent merely because both appear in one release note.

For inaccessible supplier components, state what can be tested externally and what remains dependent on supplier evidence. Vendor-controlled system auditing helps plan a bounded review. A lack of access may limit a conclusion; it is not automatically proof of a defect or a reason to omit the component without explanation.

Separate incident signals, findings and clinical consequences

An audit test failure is a finding about the examined conditions. An actual incident requires its own fact gathering and response. A potential clinical consequence should be assessed by appropriate clinical and safety owners. Keep these categories distinct so a synthetic defect is not reported as patient harm, and a genuine operational concern is not dismissed as merely a benchmark issue.

Agree an escalation route before testing. If reviewers encounter evidence of a possible current risk, they should know who receives it, how access is restricted and who can decide on service limitations. The audit team should preserve relevant facts and avoid expanding into unauthorised record searches. The organisation's responsible process determines the immediate response.

Consider whether the existing workflow can continue without the AI function. A safe exercise may examine the authorised manual process and the visibility of pending drafts. The audit should not assume that disabling the model automatically resolves every issue if drafts or interfaces remain active. Record the actual states and responsible decisions.

Make findings specific enough for clinical and technical owners

An illustrative finding might state that the tested application displayed the pre-amendment synthetic report after the source system confirmed an authorised correction, and that destination evidence did not establish presentation of the amended version. Identify the relevant configuration, views and timestamps. Do not generalise this to every patient record or claim a clinical outcome that was not examined.

Assign remediation to the correct mechanism. A presentation cache issue, an interface delivery issue and a document-association issue may need different owners. Define the expected corrected behaviour and the evidence required for closure. Retesting should include the original failure and relevant variations while checking that normal finalisation and legitimate access still work.

Report residual limitations, including unavailable destinations, narrow language coverage or insufficient operating history. A useful executive summary identifies which use decisions the evidence supports and which remain restricted or require specialist review. It should not compress clinical, privacy, security and workflow conclusions into a single reassuring accuracy percentage.

Check whether evidence transfers between sites

A multi-site healthcare group should examine which conditions are shared and which differ before relying on one site's result. Reporting templates, source interfaces, user roles and amendment routes may vary even when the same model service is used. Create a short configuration comparison and identify the differences that could affect the specific claim. A common supplier name is not evidence of an identical workflow.

For the drafting example, one site may finalise within the reporting application while another uses a separate review system. The first site's sign-off test cannot establish the second site's authority boundary without additional evidence. Reuse the relevant model-level results where justified, but test the local transition that changes the conclusion. This keeps the programme proportionate while preserving the integrity of site-specific findings.

Also record when operating evidence was collected. A site that adopted the corrected interface recently has less history than a site that used it throughout the review period. Report the difference instead of merging both into a group-wide statement of continuous effectiveness. The resulting coverage map should help leaders choose where further testing or observation is necessary.

Prepare a healthcare audit scope with the right owners

Bring the intended use, workflow map, relevant classification decisions, supplier evidence, available test environment and the records that can be reviewed lawfully and securely. Include clinical governance, operational, technical and privacy owners early enough to agree the questions and escalation route. This improves the relevance of the audit without asking the technology team to make decisions outside its responsibility.

For broader application context, Trusted's healthcare page discusses commercial healthcare AI use areas. Such material can help identify possible workflows, while the hospital's actual evidence and official requirements determine the audit. It should not be treated as proof of clinical performance or authorisation for a specific product.

Request an audit scoping discussion for a healthcare AI workflow through OpenForAI's AI auditing service with a masked workflow description and the decision requiring evidence. The engagement should produce traceable findings, practical corrective actions and clear limits, with clinical and regulatory authority remaining with the responsible parties.