Service-level design
AI Agent SLAs and SLOs
Define agent service levels around eligible tasks, accepted outcomes, transparent clocks and error budgets that lead to clear operating decisions.
An AI agent service-level objective should describe a service the business can observe: a usable result, within a defined time, for an eligible task. Model availability alone cannot establish that promise. A responsive agent may produce an incomplete packet, wait indefinitely for approval or finish after the business deadline.
This guide helps service owners, procurement teams and operations leads define AI agent SLIs, SLOs and SLAs without hiding quality behind uptime. The practical deliverable is a measurement agreement that connects AgentOps decisions to customer experience, error budgets and accountable support.
Separate the measurement, the target and the commitment
An SLI is the defined measurement. An SLO is the target for that measurement over a stated period. An SLA is an agreement specifying service commitments and consequences under agreed terms. These distinctions follow the Google SRE service-level framework; their application to your agent requires an explicit workflow definition.
For example, “percentage of eligible preflight packets delivered correctly within the deadline” is an indicator. A chosen target for that percentage over a calendar month is an objective. Contractual remedies, exclusions, reporting and dispute handling belong in the agreement. An internal objective does not automatically create a customer entitlement or a legally complete SLA.
Decide whose experience is being measured. A model provider's availability measure, an integration team's API response measure and a business user's completed-task measure describe different boundaries. Report their relationship without claiming that one vendor's promise covers all components, human decisions and external systems in the end-to-end service.
A service contract for a commercial-printing agent
Consider an illustrative agent that assembles preflight packets for commercial print jobs. It collects the approved artwork revision, identifies missing production instructions and routes a packet to the print coordinator. It does not approve colour fidelity, commit production capacity or release a job to a press.
The outcome contract distinguishes a complete packet from a correctly routed missing-information case. Both may represent appropriate handling, but they should not inflate the same completion metric. The agent outcome-evaluation guide establishes the acceptance evidence; the service-level document adds timing, measurement boundaries and operational consequences.
| Field | Proposed definition | Why it matters |
|---|---|---|
| Eligible unit | One admitted print-job revision with the required source permissions and supported file class. | Repeated attempts do not become extra tasks. |
| Start event | Admission recorded after the minimum input check. | A hidden queue cannot reset the clock. |
| Good event | Correct packet revision available to the coordinator within the agreed deadline. | A generated response alone is insufficient. |
| Exception route | A clearly identified missing-information case delivered to its owner. | Controlled handling is visible separately. |
| Measurement close | The agreed deadline plus the stated evidence-arrival allowance. | Unresolved work does not disappear. |
| Evidence owner | Service owner reconciles admission, delivery and acceptance records. | A disputed result has a resolution path. |
These are design choices, not recommended contractual numbers. The business should choose deadlines from actual production needs, supported by observed workload and capacity. A convenient round percentage can be expensive to achieve and irrelevant to the work that matters most.
Make exclusions observable before the result is known
Write eligibility rules before calculating performance. Unsupported file types can be outside the complete-packet service, but the intake service should still tell the requester what happened. Otherwise a team can improve its reported success by quietly refusing difficult work. Track admission failures and excluded volume alongside eligible outcomes.
Do not retrospectively classify a failed request as out of scope because the agent could not handle it. If the service admitted the task under the current contract, a later discovery may be a failure, a controlled exception or an agreed pause according to that contract. Preserve the original decision and reason.
Specify how customer cancellations, source changes and unavailable third parties are treated. An exclusion from a supplier remedy does not require deleting the event from the customer-experience report. Maintain both views where needed: contractual accountability and the actual service experienced by the business.
Choose clocks that reflect waiting as well as execution
End-to-end latency includes queueing, tool execution, retries and any review included in the promised service. If a separate human decision is outside the commitment, document its boundary and measure that queue separately. Do not stop a clock whenever the system hands work to a person and then describe the remaining figure as total turnaround.
Specify timezone, business calendar and whether a deadline uses elapsed or working time. A Saudi operating team may support a different calendar from an international supplier. The agreement should resolve holidays, overnight submissions and after-hours escalation without assuming that a generic weekday schedule fits the customer.
For delayed acceptance, keep pending cases visible until the measurement closes. A provisional dashboard can show known good, known bad and unresolved events. Reconcile it with late evidence under a documented restatement rule. Reporting only completed cases can make a growing backlog look like a fast, reliable service.
Calculate an error budget without concealing the denominator
Assume, purely for illustration, a monthly objective of 99% for a defined good-event indicator and 10,000 eligible tasks in the completed month. The allowed bad-event budget is 1% of 10,000, or 100 tasks. If 20 tasks miss that indicator, 20% of the budget has been consumed. This is not a proposed service promise or an observed OpenForAI result.
A live month's final task count is not yet known. A dashboard may show a provisional budget using observed traffic or a forecast, but it must label which basis it uses. Do not mix a forecast denominator with a final-month claim. Volume changes also affect how quickly a request-based budget is consumed.
The Google SRE guidance on SLO alerting defines burn rate relative to the allowed error ratio and explains multiwindow alerting. With the illustrative 1% allowance, a 5% bad-event ratio over a specified recent window is a burn rate of 5. That ratio alone does not say exactly how many calendar hours remain before the monthly budget is exhausted.
Use a longer window to establish sustained impact and a shorter window to confirm it is still occurring. Choose windows and thresholds for your task volume and response capacity. Low-volume workflows may need event-level escalation for consequential failures because ratios based on very few tasks are unstable.
Keep safety constraints outside a tradable budget
An error budget supports reliability decisions; it is not permission to perform a forbidden action. An unauthorised instruction to release a job to a press, sensitive-data exposure or access-control bypass needs its own incident rule. A healthy monthly average must not neutralise a hard policy breach.
Similarly, a high completion rate cannot compensate for an unacceptable error in the wrong category. Report quality, timeliness and controlled handling separately, then define any composite indicator explicitly. If a task must be both correct and on time to count as good, make that conjunction visible rather than multiplying unrelated percentages.
Connect the budget policy to agent release control. A depleted budget may pause nonessential changes while reliability work continues. State who can grant an exception, what evidence is required and when it expires. Otherwise “error budget” becomes a dashboard label with no effect on operating decisions.
Write the supplier agreement around evidence and remedies
Specify the source of truth for task counts, timestamps and acceptance. State the reporting cadence, data access, dispute period and process for correcting measurement defects. If either party can change the denominator unilaterally, a precise-looking percentage will not prevent disputes.
Separate acknowledgement, investigation, mitigation and restoration targets. A supplier responding quickly to a ticket does not establish that the agent service has recovered. The on-call and escalation model should identify who can act when the model provider, integration service or customer-owned system is unavailable.
Describe remedies and exclusions with the people responsible for the contract, using the real operating boundary. Avoid copying a cloud infrastructure SLA into an agent engagement. A provider can only commit credibly to measures it can observe and influence, while the customer still needs ownership for the complete process.
Aggregate task evidence consistently
Combine segment counts before calculating an overall good-event ratio, or publish the weighting rule when a weighted score is intentional. Averaging percentages from very different task volumes can distort the service picture. Keep the segment results available so a large easy workload cannot conceal a small consequential failure class.
Count a logical task once under the selected indicator, even when it has several attempts. A retry may improve its eventual outcome while worsening latency and consuming capacity. Preserve those separate effects rather than counting every successful attempt as another good business task.
Validate the measurement before making the promise
Run the proposed SLI in observation mode and inspect individual cases. Reconcile task counts against the business intake system. Test duplicate admissions, delayed records, cancelled revisions and missing timestamps. Confirm that an unresolved task remains visible after a reporting cutoff and cannot become a success through a retry-counting defect.
Review segment results by workload class and language where those change service behaviour. A strong overall result may conceal a weak Arabic file class or a late-afternoon queue. Use production monitoring for collection and the AgentOps scorecard for the agreed management view, preserving the same definitions across both.
Before approval, have the service owner explain one good event, one bad event, one excluded request and one unresolved case from source records. If the examples cannot be reproduced, refine the measurement. Discuss AI agent service-level design with a masked task sample, the current business deadline and the boundaries each supplier controls.
FAQ
Your questions, answered.
What is the difference between an AI agent SLO and an SLA?
An SLO is an operating target for a defined indicator over a stated period. An SLA specifies commitments, measurement terms and consequences agreed between parties. An internal target does not automatically create contractual remedies. First establish eligible tasks, good events, timing and evidence, then have the responsible commercial and legal owners define the agreement.
Should human approval time count towards an agent service level?
It depends on the promised service boundary. If the commitment is end-to-end delivery including review, that waiting time belongs in the measure. If approval is a separate customer responsibility, document the boundary and track its queue independently. Do not silently remove human waiting and label the remaining duration total turnaround.
How do we handle tasks still pending at the reporting cutoff?
Show them as unresolved under the published measurement rule rather than dropping them from the denominator. Define the deadline, evidence-arrival allowance and process for revising provisional reports. A completed-only view can hide an ageing backlog. Preserve admission records so later acceptance can be attributed to the correct task and cohort.
Does an error budget permit occasional unauthorised agent actions?
No. Reliability budgets support decisions about ordinary service failure and change capacity. Prohibited actions and material security or policy breaches need independent escalation rules. A strong monthly average cannot offset an unauthorised action. Define hard constraints separately from the quality and timeliness indicators used for the service budget.
Can we adopt our model provider’s SLA for the complete agent?
A model provider’s agreement covers its defined service, not automatically your queues, tools, data, integrations, human review or business outcomes. Map each responsibility and measure the end-to-end experience separately. Contractual exclusions can coexist with a customer-experience report showing the real impact of an external outage.
How should low-volume AI agents use SLO alerts?
Percentages based on a few tasks can fluctuate sharply, so combine windowed indicators with individual consequential-event rules and backlog-age checks. Choose thresholds using the task volume and response capability. Report the event count with every ratio, and avoid importing high-traffic alert settings without testing their behaviour on your workload.