Healthcare operations
AI Agents for Healthcare
Design an administrative referral agent with evidence-based closure, clinical boundaries and bilingual acceptance tests.
A patient can leave a consultation with an appropriate care plan and still miss the next administrative step. A referral remains in a queue, the booking team cannot see whether an instruction was approved, or an unanswered message is mistaken for a completed follow-up. For a Saudi hospital group or outpatient network, an AI agent is most useful when it helps staff close these gaps without quietly acquiring clinical authority.
This guide is for patient-access directors, clinical operations leaders, digital-health teams and the clinicians who must approve the boundary. Its proposed starting point is an administrative referral coordinator: it reads authorised instructions, assembles missing information, prepares the next task and shows what remains unresolved. That scope is narrower than a clinical decision system, but it still requires careful design because an administrative failure can affect care.
Choose the care-pathway gap before choosing the model
Start with a specific transition: an approved referral reaching a booking team, a pre-visit document request reaching the right patient, or a completed appointment returning a status to the referring service. Map who currently notices a failure. If the answer is “whoever checks the spreadsheet”, the project needs an accountable queue owner before it needs an agent.
For each transition, distinguish the task the agent may prepare from the action another system or professional authorises. A referral coordinator might extract the destination clinic from a signed instruction and draft an outreach task. It should not infer a new speciality from symptoms, rewrite urgency or decide that follow-up is unnecessary. An ambiguous instruction belongs back with the responsible team.
The same distinction applies to patient messages. An agent can explain an approved administrative process, but a newly reported symptom should enter the organisation’s clinically approved escalation pathway. A generic disclaimer does not turn an unrestricted clinical conversation into a controlled administrative workflow. Document these edges in the project scope and make them visible in the staff interface.
A referral has more states than “open” and “done”
The most consequential design choice may be the state model. A generated message is not a delivered message; a delivered message is not an acknowledgement; and an acknowledgement is not a completed appointment. Each state should require evidence from the relevant source, with a named owner when that evidence does not arrive.
| State | Evidence | Agent boundary |
|---|---|---|
| Instruction available | Signed instruction, patient reference, author and version | Retrieve; do not reinterpret clinical intent |
| Task prepared | Required booking fields and missing-item list | Draft the task and flag uncertainty |
| Outreach authorised | Approved recipient, channel and message version | Release only through the agreed approval rule |
| Patient acknowledged | Recorded response or staff confirmation | Do not infer acknowledgement from delivery |
| Appointment confirmed | Booking-system confirmation identifier | Report the actual booking outcome |
| Referral closed | Completion evidence or authorised closure reason | Never close merely because retries ended |
Keep patient choice distinct from an operational failure. A patient declining an offered appointment, an unreachable telephone number and a full clinic are different exceptions. Each calls for a different owner and response. Combining them as “not booked” prevents the organisation from seeing whether capacity, contact details or the handoff itself is the problem.
Give every source a limited job
The electronic health record should establish the approved instruction and relevant identity reference. The scheduling system should establish available slots and booking confirmation. The communication service should establish delivery status. The agent’s working notes should establish what it attempted, not become a competing clinical record.
Match patients using the organisation’s authorised identity workflow, not approximate names alone. Arabic and English names, transliterations, dependent records and shared family contact numbers create cases that need explicit handling. If the identity match is unresolved, the safe operational outcome is a held task for verification, rather than a confident merge.
Provide the minimum information needed for the task. A booking coordinator may need the referral destination, approved preparation instructions and contact permission; it may not need an unrestricted longitudinal record. Separate staff roles and test what each can retrieve. The PDPL implementation guide covers the broader processing assessment; the healthcare project must additionally establish its own clinical, contractual and sector-specific requirements.
Integrations also need failure semantics. If a booking request times out, query its result before trying again. A second attempt must not create a duplicate appointment. If an instruction changes after an outreach draft was approved, invalidate that draft and request review. A reminder based on a superseded instruction is not made safe by fluent wording.
Keep intended medical purpose under review
The Saudi Food and Drug Authority publishes guidance for AI and machine-learning-enabled medical devices and software as a medical device. These are relevant references when a proposed feature moves into medical-device territory. Calling software an administrative assistant does not settle its classification; the actual intended purpose and functionality need qualified assessment.
Build a change-review trigger into the project. A request to prioritise patients by symptoms, interpret a result or recommend treatment is a material scope change, even if it looks like an additional prompt. The clinical sponsor, regulatory owner and technical lead should reassess it before development. Keep approved educational material separate from patient-specific clinical recommendations.
Human review also needs capacity. If every uncertain task goes to the same clinician, the system can create a hidden backlog. Agree which issues a booking supervisor can resolve, which require a clinical team, and how time-sensitive items enter the existing escalation process. See the human-oversight design guide for review permissions and escalation ownership.
Worked design: a referral that changes after outreach
Illustrative scenario, not a reported deployment: an outpatient network tests the coordinator on 120 de-identified historical referral journeys. The approved instruction initially directs the patient to Clinic A. Staff approve an outreach draft, but a clinician then updates the instruction to Clinic B before the message is released.
The expected behaviour is to detect the changed instruction version, block the pending message and return the task for review. It is insufficient to append the new instruction to the old conversation and hope the agent notices. The release operation must check the current version explicitly. The test record should retain the original draft, the change event, the blocked action and the reviewer’s replacement.
Assume, purely for planning, that 18 of the 120 journeys contain unresolved contact or booking exceptions. The pilot should report how many of those 18 were correctly identified and assigned, how many were missed, and how many normal journeys were incorrectly escalated. A high average answer score could conceal all 18 exceptions. The useful denominator is the relevant exception class.
Do not turn this retrospective test into a claim that patients received faster care. Historical replay can show whether the workflow would recognise specified conditions. A controlled live pilot is needed to observe staff workload, successful contact, patient experience and actual completion. Even then, case mix and clinic capacity may explain changes that cannot fairly be attributed to the agent.
The acceptance artifact: a closure evidence register
Ask the delivery team for a sample register before approving the pilot. Each row should identify the referral, source instruction version, current state, evidence supporting that state, next owner, next due action and permitted closure reason. Use pseudonymised examples in procurement and demonstrations.
Add a challenge set that tries to produce false closure: an undelivered message, an appointment request without confirmation, a cancelled slot, a patient identity collision, an amended instruction and a reply containing a clinical question. Require reviewers to explain the expected outcome before running the system. This prevents the acceptance standard being adjusted to suit a persuasive demo.
Assess Arabic and English separately where both are used. Check patient-friendly language, date clarity, clinic names, numerals and whether translation changes an instruction’s meaning. A bilingual reviewer should compare the released message with the approved source, not merely rate the writing as natural. The Arabic-native agent guide explains language evaluation beyond translation.
The pilot’s measures should include false closure, wrong-recipient attempts, unowned overdue tasks, review time and confirmed administrative completion. Treat any unauthorised clinical action as a stop condition defined with the clinical sponsor. Track message volume only as an operating measure; more messages can mean more friction.
Commission a service that staff can safely interrupt
Roll out by pathway and team, not by giving the agent access to every record at once. Start in replay, then staff-visible preparation, then an explicitly approved release mode if the evidence supports it. Keep a route for staff to pause a case, correct the source and resume with a recorded reason. Recovery should not require editing hidden prompts or asking a developer to repair the conversation.
Agree how changed clinic instructions, message templates and integrations will be reviewed after launch. The service owner needs a queue-health view and a way to detect stale evidence; the technology owner needs integration and permission monitoring. The testing and evaluation guide helps turn these conditions into release evidence rather than a one-off demonstration.
Open for AI’s custom AI agent development service can be scoped around this kind of defined workflow; integrations, deployment and operating support must be agreed for the project. To discuss a healthcare administration agent, bring a redacted referral journey, its authorised state transitions, a sample failed handoff and the roles permitted to resolve it. Those materials reveal whether the opportunity is better routing, better source data or a bounded agent that connects the two.