AI Governance for Healthcare

AI governance in healthcare should establish which clinical or administrative purpose is permitted, who remains accountable for its consequences and what evidence is needed before that purpose changes. An assistant that reformats approved discharge instructions and a system that proposes treatment changes need different assessments, even if they share the same interface and model.

For Saudi providers, governance setup should connect clinical leadership, patient safety, privacy, information security, technology and operations. This guide is for hospital executives, clinical informatics teams and governance owners building that connection. It uses an illustrative discharge-document workflow to show how authority, evidence and change control can be made practical.

Classify the intended purpose before choosing the review route

Describe what the system is intended to do, the information it receives and how its output affects care or administration. Avoid relying on labels such as non-clinical, productivity or assistant without examining the workflow. A document tool can influence patient understanding or a clinician's next action.

The SFDA's software guidance links medical-device status to intended use. Its AI/ML medical-device guidance addresses relevant device purposes and regulatory considerations. Obtain a qualified assessment of the actual product and current requirements; this article does not classify a particular tool.

Separate the initial purpose from plausible future features. A supplier roadmap that adds recommendations, prioritisation or automatic record updates can cross an important boundary. The inventory should describe the authorised function today and the triggers that would require a new decision.

Use AI use-case classification to route review according to the consequences and uncertainty of the proposed use. Keep internal risk tiers distinct from any regulatory device classification.

Make clinical and operational authority explicit

Name the clinical owner for clinical content, the operational owner for the workflow and the technical owner for the service. Define who can approve the intended use, restrict it, stop it and authorise restart. These roles can cooperate without assuming that one technical approval settles every question.

Specify the reviewer's responsibility at the point of use. “Human in the loop” is incomplete if the person lacks time, source access, relevant competence or the authority to reject the output. The interface and staffing model must make the review possible.

For a discharge draft, identify who checks the patient's episode, the current approved instructions and the final document before release. The AI output should not become an authoritative record merely because it has been generated or copied into a familiar template.

The AI responsibility map should also cover nights, leave and service disruption. A named owner without a backup or escalation route can leave an urgent operational decision unresolved.

Worked example: discharge drafting expands into medication suggestions

Consider an illustrative provider authorising an assistant to draft a discharge letter from clinician-approved source material. The permitted function is to restate and organise that information for professional review. It does not select treatment, reconcile conflicting orders independently or propose medication changes.

A later supplier update offers a feature that suggests adjustments when it detects a possible inconsistency. The team cannot treat this as a cosmetic improvement to the existing drafting tool. The proposed purpose, consequences, required expertise and potentially relevant regulatory assessment have changed.

The governance process records the new function and prevents it from being enabled under the old approval. Clinical and regulatory specialists assess the proposed use, while technical and privacy teams review the changed processing. The original drafting service may continue only if its existing approved boundary remains effective and separable.

Illustrative authority boundaries for a discharge assistant

Function	Proposed operating boundary	Review implication
Organise approved instructions	Draft only, with source trace and professional review	Evaluate faithful restatement and workflow usability
Resolve conflicting clinical sources	Flag the conflict for the authorised clinician	Do not let the assistant invent a resolution
Suggest treatment changes	Outside the original approval	New clinical-purpose and regulatory assessment
Release a final patient document	Authorised human process confirms the final version	Verify identity, episode and approved content

The example defines a governance boundary, not a clinical treatment protocol. The provider's responsible professionals determine the appropriate workflow and safeguards for its actual service.

Preserve source authority and episode identity

Map which records the assistant may use and how it distinguishes current, superseded, draft and cancelled information. A technically successful retrieval can still select the wrong encounter or a document written before the final clinical decision.

Carry source identifiers and relevant version information into the review view. Reviewers should be able to see why a statement appears in the draft and whether it remains supported. An unsupported sentence should be identifiable without searching an entire record manually.

Handle missing and conflicting sources explicitly. A blank field, unavailable attachment and confirmed absence of an instruction are different states. The assistant should not convert missing information into reassurance or fill a gap using general medical knowledge when the approved task is faithful drafting.

Test identity mismatches and duplicate names using appropriately controlled examples. The system should preserve the patient's episode boundary through retrieval, generation, review and release. A correct sentence about the wrong episode is still an unacceptable output.

Design bilingual review around clinical meaning

Arabic and English evaluation should examine meaning, not only fluency. Review whether negation, timing, conditions, uncertainty and the status of an instruction remain intact. A polished translation that changes a recommendation into a completed action can mislead the reader.

Use reviewers competent in the relevant clinical and language context. Define how terminology disputes are resolved and which source instruction is authoritative. Avoid assuming that a vendor's claim of Arabic support demonstrates suitability for the provider's documents.

Include abbreviations, mixed-language source text and amended instructions in the evaluation set. Test whether the draft preserves important distinctions when the final approved instruction differs from an earlier note. Record errors by their consequence rather than grouping all language issues as cosmetic.

Keep patient-facing clarity separate from clinical authorisation. Simplifying language can improve understanding, but the simplification still needs to preserve the approved meaning and go through the appropriate professional review.

Evaluate the complete workflow before expanding access

Construct a challenge set with deliberate variation. An illustrative set might contain 30 prepared cases: 12 routine drafts, eight amended-instruction cases, six bilingual cases and four cancelled-discharge cases. These groups are distinct in this example. The selection tests important conditions; it does not estimate an organisation-wide error rate.

Define expected handling before evaluation. A cancelled discharge should not produce a releasable final letter. A conflict should be escalated. A current instruction should remain distinguishable from an earlier version. Reviewers should know which errors require immediate restriction and which require repair before expansion.

Measure the work of the human reviewer as well as generation. Track verification burden, missed changes, inappropriate acceptance and the ability to find sources. Faster drafting does not establish a benefit if checking the output takes longer or important errors become harder to notice.

Separate draft usefulness, clinical safety, privacy and operational reliability in the decision. A high score on one dimension does not cancel an unresolved condition on another. The impact assessment should connect those findings to the permitted scope.

Limit access to the information needed for the task

Map users, service identities, repositories and support access. The assistant should receive only information required for the approved purpose and authorised audience. A clinical role in one service does not automatically justify access to every patient record in the organisation.

Inspect derived copies, including prompts, generated drafts, logs and exported evaluation examples. A secure source system does not establish that each downstream copy has the same protection. Privacy and security owners should define the required handling for the actual architecture.

Use PDPL-aware AI governance to connect the processing assessment, supplier arrangement and rights handling to operating controls. Do not turn an AI approval into a blanket authorisation for secondary training or unrelated analytics.

Prepare evaluation material through approved methods such as synthetic cases, appropriately protected extracts or controlled review. The governance team should not create a new uncontrolled patient dataset merely to make an assessment convenient.

Ask suppliers for evidence tied to the clinical purpose

Request a clear intended-use statement, limitations, change process and evidence for the function being purchased. Distinguish regulatory status where applicable from a general security certificate or a demonstration using unrelated documents.

Understand whether the supplier can change model behaviour, prompts, retrieval or features without local notice. Establish the information and controls needed to assess significant changes before they affect the approved workflow. A familiar product name can conceal a materially different configuration.

The supplier evidence requirements should include the ability to investigate an unexpected output while protecting patient information. Identify who can access logs, how evidence is preserved and what assistance is available when a clinical or privacy concern arises.

Where broader implementation context is useful, Trusted Data's healthcare AI page describes sector applications. A sector overview does not establish the safety, regulatory status or suitability of a particular product for this provider.

Build a stop and restart process that clinical teams can use

Define practical triggers for pausing the affected function: cross-patient content, unsupported clinical changes, loss of source traceability, an uncontrolled feature update or failure of required review. The response should identify the affected scope rather than forcing an all-or-nothing decision without evidence.

Provide a known fallback to the established documentation process. Staff should be able to complete necessary work while the AI service is restricted. Test the fallback under realistic staffing and access conditions, including how pending drafts are labelled and handled.

Follow the organisation's patient-safety, privacy and technology incident routes as applicable. The AI incident response plan should connect them and preserve a common timeline. Do not overwrite the original generated draft when correcting the final record; maintain the appropriate evidence under the provider's record rules.

Restart requires an authorised decision supported by the cause analysis, corrective action and relevant retesting. A supplier's statement that the model was improved is not sufficient evidence that the provider's particular failure mode is resolved.

Monitor the boundaries after deployment

Track use outside the approved task, review bypass, unsupported statements and repeat corrections. Look at near misses and cases where staff abandoned the draft. Successful document counts alone will not show whether the service is creating hidden checking work.

Review changes in source systems and clinical documentation practices. A revised discharge template or new source field can affect the assistant even when the AI model is unchanged. Keep the service owner connected to the teams that control those sources.

Use sampled review to challenge continuing assumptions. Include different departments, language patterns and shifts where they fall within the approved scope. Do not generalise evidence from one narrow pilot to every specialty or patient population.

Report unresolved conditions to the accountable clinical and executive owners. The oversight report should make clear whether a problem affects content fidelity, operating capacity, privacy or scope, because each requires a different decision.

Trace corrections after a discharge draft has been used

A governance design should cover the moment after a reviewer has accepted a draft as well as the review itself. Suppose a clinician corrects the source record after a discharge summary has been prepared. The workflow needs to identify whether a draft is still pending, whether an authorised document has been released and which downstream recipients or systems may hold the earlier version. The clinical and records owners determine the appropriate correction process.

Do not let the assistant silently replace the historical record or send its own explanation to a patient. It should expose the source change and the documents potentially affected to the authorised team. That team determines clinical significance, required communication and record handling. A language improvement and a material correction may require different treatment, so the system must preserve the actual change rather than flatten everything into a new version number.

In the discharge example, keep patient and episode identity attached to the correction. A patient can have several encounters, and a corrected observation may relate to one episode only. Test whether the workflow identifies the right draft and prevents an update from being copied into another encounter. The test concerns record integrity and routing; it does not prescribe a clinical decision about the corrected information.

Make the handoff visible to the next shift. The responsible reviewer should be able to see that a correction is pending, what source changed, who must assess it and whether any communication has already occurred. A private message to the original reviewer is not a durable operating record. Provide a designated route when that reviewer is unavailable.

Assess whether reviewers can perform the approved task

Human review needs more than a named clinician in the workflow. Observe whether the reviewer can access the relevant source, distinguish generated text from approved content and make a correction without losing the link to the encounter. Ask staff to explain the limits of the assistant and the point at which a proposed new function needs reassessment.

Use the medication-suggestion expansion as a training challenge. The reviewer should recognise that the proposed function differs from drafting an existing approved plan and refer it to the appropriate clinical, regulatory and governance process. The exercise should not ask staff to validate a clinical capability through a few examples. It tests recognition of the scope boundary and the ability to keep an unapproved feature out of routine use.

Record workflow obstacles as design findings. If the source is hidden behind another login, a correction is cumbersome or the output appears authoritative before review, address those conditions. Asking staff to be more careful does not remove a practical barrier to the oversight on which approval depends. The service owner should verify the revised workflow with representative users before expanding access.

Include the receiving team in acceptance. A corrected document may be understandable to its author but ambiguous to the next ward or service. Ask an authorised recipient to identify its status, source context and required next step. Resolve any misleading presentation through the clinical document process, keeping the assistant's role and the responsible clinician's decision clearly distinguishable.

Prepare a healthcare governance handover

The handover should include the approved purpose, excluded functions, authority map, evidence, supplier commitments, access conditions, review instructions, incident route and change triggers. Give operating teams a usable description of what they may do and what requires escalation.

Run a tabletop exercise using the proposed medication-suggestion update. Ask product, clinical, privacy and service owners to identify who detects the change, who prevents unauthorised activation and what evidence is needed for a new decision. Resolve gaps before expanding the service.

For AI governance setup, bring the actual clinical or administrative workflow and current approvals. Discuss healthcare AI governance setup with Open for AI to define decision routes and operating evidence around the provider's accountable professionals. The aim is a maintainable governance system that preserves the intended purpose and makes changes visible before they affect patients.