AgentOps Maturity Model

An AgentOps maturity model describes how reliably an organisation can operate its agents, demonstrate its controls and improve them over time. It should help choose the next capability to build. It should not reward autonomy for its own sake or turn a collection of tools into a claim of operational excellence.

This guide proposes a practical four-state model for service owners and operating leaders. It is an original planning framework, not an official certification, a regulatory rating or a promise that a higher label makes every agent suitable for every task. Use it within AgentOps to compare current evidence with the operating capability the service actually needs.

Assess capability rather than the sophistication of the demo

A sophisticated agent may still depend on one developer, lack a verified outcome or have no usable recovery procedure. A simpler agent can be well operated with explicit limits, dependable review and tested response. Judge the operating evidence, not the number of tools, agents or automated actions.

Define the unit being assessed. A particular service, deployment or portfolio can have different operating conditions. Do not assign a company-wide maturity label from one carefully supported agent. Record the scope, evidence period and responsible owners so the assessment can be repeated.

NIST's AI RMF profile approach supports describing current and target states in context. The stages below are this guide's proposed operating progression; they are not NIST-defined levels. The target should follow consequence, workload and organisational need rather than a desire to claim the highest stage.

Four operating states, each requiring evidence

Proposed AgentOps capability progression

State	What is demonstrated	Typical evidence	Next question
Mapped.	The service and its responsibilities are identifiable.	Purpose, owners, deployments, tools and permitted actions.	Do the intended controls actually work?
Demonstrated.	Important operating controls work in defined exercises.	Outcome evaluation, denied-action test, pause and recovery rehearsal.	Can ordinary operators sustain them through real variation?
Sustained.	The service maintains the controls through normal work, changes and absence.	Repeated operating evidence, receiver-led response and reviewed exceptions.	Can the team improve the system without weakening its boundaries?
Learning-led.	Changes are selected from evidence and their effect is verified.	Closed corrective actions, comparable outcome evidence and maintained tests.	Which emerging conditions require a new target or control?

Use “unverified” when evidence is insufficient for a state. It is not a failure grade or a reason to invent missing records. A team can know the right design and still need to demonstrate it. Conversely, buying an observability platform does not automatically move the service to demonstrated or sustained operation.

Assess several dimensions independently

Use dimensions that correspond to operating decisions: outcome evidence, action authority, human review, monitoring and response, release and recovery, resource control and learning. Each dimension may occupy a different state. A service can have sustained cost reporting while recovery remains unverified.

For outcome evidence, ask whether the accepted task is defined, tested and consistently observed. For authority, inspect the enforced action boundary and changes to it. For human review, examine the decision rights, evidence, capacity and ability to reject a proposal.

For response and recovery, check whether the assigned operator can contain and reconcile the service under realistic conditions. For change, ask whether releases preserve the evidence and authority they depend on. For resource control, establish whether budget and capacity limits are enforced or merely reported after the fact.

Do not average a critical unverified control into a reassuring composite. Identify the minimum capabilities needed for the approved scope and make material gaps visible. The readiness checklist remains the release gate; a maturity profile does not override it.

Worked progression: museum collection-description drafts

Consider an illustrative museum agent that prepares draft descriptions from approved collection records. A curator reviews attribution, context and uncertainty before publication. The agent cannot invent provenance, settle ownership questions or publish a description without the required approval.

At the mapped state, the team identifies the agent, collection sources, curator decision, deployment owner and publication boundary. This creates accountability, but does not yet prove that the agent preserves uncertainty in an object record or that the operator can stop an unsuitable draft.

At the demonstrated state, controlled cases include an object with incomplete attribution, a changed source record and a request outside the approved collection. The team observes the correct review or hold, checks the action boundary and rehearses containment. These are specific proofs, not evidence that every future case will be correct.

At the sustained state, routine records and exceptions remain traceable through curator absence, source revisions and approved releases. An authorised substitute can operate the review arrangement, and the service does not depend on the original developer interpreting every ambiguous case.

At the learning-led state, the team notices recurring ambiguity in a source field, improves its definition with the collection owner and verifies the effect on comparable drafts. The curator can still retain final publication approval. The progression improves evidence and repeatability without requiring the agent to become more autonomous.

Build a progression card for the next capability

Illustrative progression card for the museum service

Dimension	Current evidence	Gap	Next proof
Outcome evidence.	Curators review drafts, but reasons for holds are inconsistent.	Unclear distinction between source ambiguity and a draft defect.	Agreed dispositions applied to a controlled review set.
Operating ownership.	The original developer resolves every queue interruption.	Receiving operators have not demonstrated recovery.	A receiver-led exercise using approved access and the current procedure.
Learning.	Recurring source issues are discussed informally.	No verified link from an improvement to changed outcomes.	A documented change with comparable before-and-after evidence and limits.

Assign an owner and a decision date appropriate to each proof. Avoid writing “reach the next level” as the action. The useful commitment is the capability to demonstrate and the operating consequence it addresses.

Choose priorities from consequence and dependency

Start with gaps that prevent safe or accountable operation of the current scope. An unknown action permission can matter more than a sophisticated portfolio dashboard. A missing business owner can block the interpretation of evaluation results, so ownership may need to precede metric refinement.

Identify the order in which improvements depend on each other. Define the accepted outcome before optimising its unit cost. Establish the receiving operator's access before scheduling a recovery exercise. Use the assessment method to gather the evidence rather than assigning stages through self-perception alone.

A smaller organisation may combine roles, but should keep the decisions explicit. A larger organisation may need shared capabilities and local service owners. Neither structure is automatically more mature; the question is whether the required operating capability is demonstrated for the service.

Distinguish maturity from current performance

A scorecard reports what the service is doing over a defined period. A maturity profile asks whether the organisation can repeatedly operate, explain and improve that behaviour. They inform each other but are not interchangeable.

A mature process can reveal disappointing quality and choose to hold work. An immature process can report excellent results from selected examples while hiding unsupported tasks. Read performance together with evidence coverage and operating capability, rather than rewarding only a favourable headline.

Use incidents as evidence without treating every incident as proof of low maturity. The response, containment, learning and recurrence matter. The post-mortem process can demonstrate improvement when actions are closed with evidence; a stack of unimplemented review documents cannot.

Reassess when the operating conditions change

New tools, users, source relationships, languages, hosting arrangements or response coverage can invalidate earlier evidence. A team reorganisation can also remove the person who understood a critical procedure. Record which dimensions need renewed proof after a material change.

Retain the earlier profile and explain the change. Moving from sustained to unverified in a new scope is an honest account of evidence, not necessarily organisational regression. Do not protect a label by pretending that an old exercise covered a newly consequential action.

Use the agent inventory to connect the profile to current deployments and owners, and the handover process to transfer capability when responsibilities change. This keeps the maturity record tied to operation rather than a periodic presentation.

Record the conditions behind a sustained claim

Describe the workload and disturbances observed during the evidence period. A quiet period with no releases and the original developer always present does not test the same capability as ordinary operation through changed sources and staff absence. State what actually occurred and which conditions were covered only by an exercise.

This makes profiles more comparable without inventing a universal minimum number of weeks. The relevant question is whether the evidence supports the capability being claimed. If a rare but consequential condition has not occurred, a controlled rehearsal may provide useful evidence with explicit limits. Keep that distinction visible instead of treating every successful exercise as proof of sustained production experience.

Use the model to make a bounded improvement decision

Document the assessed scope, current evidence by dimension, target capability, priority gaps and next proofs. State what is not assessed and why. If the service only needs a bounded draft workflow, a strong human approval arrangement can be a deliberate target rather than a temporary embarrassment.

Discuss an AgentOps maturity profile with one service's ownership record, recent operating evidence and an improvement decision. The model is useful when it turns a vague ambition into a specific capability the team can demonstrate and sustain.