Authorised boundary testing
AI Security Boundary Testing
Test whether identity, tenant, retrieval, cache and tool boundaries prevent unauthorised effects using controlled defensive evidence.
AI security boundary testing examines whether an application enforces the limits between identities, tenants, data, tools and external effects. The question is not simply whether a model refuses an inappropriate request. It is whether the systems that retrieve information, execute tools, cache results and expose records prevent an unauthorised effect under the agreed test conditions.
This guide is for security owners, technology leaders and auditors planning authorised defensive testing. Its hypothetical example is a multi-tenant legal-matter status assistant. The assistant may summarise permitted case-status information for a verified user. It must not expose another tenant's matters or expand access to privileged records. Tests use synthetic matters and controlled identities; no real client secrets are required to demonstrate the method.
NIST SP 800-115 provides guidance on planning and conducting technical security tests and interpreting their limitations. It is a security-testing reference, not an AI certification. The procedures below are an original, bounded approach within AI auditing services, and must be authorised for the particular environment and operations.
Define what the boundary is meant to protect
Start with assets and permitted effects. In the example, protection may apply to matter existence, client identity, status text, attachments, internal notes and tool actions. A system can disclose that a confidential matter exists without revealing its full contents. The expected control should therefore specify which information is restricted, rather than treating a hidden document body as the only protected asset.
Identify the principals making and executing requests. A user may be authenticated through an identity service while a backend retrieves documents using a broad service credential. Authentication establishes who presented credentials; authorisation determines what that identity may do. The audit needs to follow both, including any translation from user identity to application role or tenant scope.
List the enforcement points. They may include a gateway, application policy layer, retrieval filter, tool executor or destination service. A model instruction can express the intended rule but should not be mistaken for the mechanism that enforces access. Record where a denied request is stopped and what evidence confirms that the protected operation did not proceed.
Agree rules of engagement before testing
Written authorisation should identify the system owner, environments, accounts, permitted test classes, operating window and stop conditions. Specify what testers may inspect and which actions are excluded. Permission to review a chatbot does not automatically authorise testing every connected provider, tenant or production integration. Resolve those dependencies before execution.
Use synthetic records with distinctive, non-sensitive markers and approved test identities. Keep a fixture inventory showing which identity may access each item. If masking is used instead, verify that the remaining data cannot expose unnecessary client information. The evidence handling guide addresses authority, minimisation and custody of the records collected during testing.
Define a contact and containment process for unexpected exposure or service disruption. Testers should know when to stop, what minimal evidence to preserve and who can authorise continuation. Do not continue collecting more protected information merely to make a finding look dramatic. One well-evidenced boundary crossing may be enough to establish the issue under the tested conditions.
Build an identity-to-operation matrix
Map each test identity to the object and operation it may use. Tenant separation, role separation and object-level restrictions are different dimensions. A staff member may have access to some matters within one tenant but not all of them. Testing only a tenant administrator and an anonymous user can miss the boundary between ordinary users with different assignments.
| Identity and context | Permitted operation | Restricted operation | Evidence location |
|---|---|---|---|
| Tenant A assigned user | Read status for assigned synthetic matter A1. | Read unassigned matter A2. | Application decision, retrieval context and rendered answer. |
| Tenant B assigned user | Read assigned synthetic matter B1. | Discover or read tenant A matter data. | Search results, snippets, citations and response. |
| Read-only user | View permitted status information. | Change a matter or trigger an unauthorised workflow. | Tool executor and destination record. |
| User after role removal | Use only rights remaining under the documented session policy. | Retain removed access beyond the agreed rule. | Identity state, cache and supported session behaviour. |
| Support operator | Use explicitly authorised support metadata. | Read client content through unrestricted diagnostic records. | Support interface and log-access controls. |
Approve the expected results with the system owner before running the cases. The matrix should represent actual policy, including justified exceptions. An auditor should not classify a permitted support workflow as a vulnerability simply because it is unfamiliar, nor accept broad access without evidence of the authority that permits it.
Test the application boundary, not only the conversational response
Inspect what happens before and after generation. A model may refuse to show a restricted matter after the retrieval layer has already supplied its content. That observation differs from a request denied before retrieval. Conversely, a model may generate an inappropriate statement about access without any protected data being retrieved. Separate text behaviour from confirmed system effects.
For each controlled case, preserve the request identity, target fixture, policy decision, relevant intermediate evidence and final output. Where the system exposes no intermediate trace, the audit can assess observable output but should qualify the inference about internal enforcement. A clean visible response is not proof that every internal boundary operated correctly.
Check alternative presentation paths such as citations, downloadable summaries, search suggestions and error messages where they are in scope. An answer body can be protected while a title, identifier or export contains restricted information. This is why the asset definition should include metadata and derived content, not only the original document.
Examine untrusted content and tool authority
External documents and retrieved material may contain text that resembles instructions. Use benign synthetic fixtures that ask for a departure from the approved workflow, then observe whether the trusted execution controls maintain the boundary. The purpose is to test resistance to unauthorised influence, not to collect real secrets or provide an unrestricted exploitation exercise.
The current OWASP LLM security guidance is a useful threat-selection reference. Its presence in a report is not evidence that a particular application has been tested against every relevant threat. Choose cases based on the system's data flows and permissions, and state the coverage actually achieved.
Where the assistant can call tools, compare the proposed operation with the action accepted by the executor. A rejected proposal may demonstrate a functioning control. An accepted operation needs destination evidence to establish its effect. The agentic AI audit guide examines approval binding, delegation and uncertain outcomes in more detail.
Challenge cache and session boundaries
Caching can create a separate access path from the original retrieval. Test whether cached content remains appropriately scoped to the user, tenant, role and other relevant conditions. A cache key containing only the text of a question may be insufficient for a service whose answer depends on permissions, but the audit should examine the actual implementation rather than assume the design.
In a controlled example, an authorised Tenant A identity requests synthetic matter A1 and populates a cache. A Tenant B identity then makes an equivalent request under its own session. Inspect whether the second response contains A1's marker, metadata or derived summary. Keep the expected denial and the exact fixture permissions documented so the result has an unambiguous reference.
Test role changes according to the documented policy. Some systems require a new session or token refresh; others propagate changes differently. Establish what the service promises and whether that promise is appropriate for its use. The auditor should not invent a universal revocation time, but should identify undocumented or unenforced behaviour that creates an exposure.
For retrieval-specific access paths, coordinate with RAG system auditing. The security test establishes the boundary behaviour, while the RAG review can investigate source permissions, indexing and derived copies. Keep the two conclusions aligned without claiming that one small test proves the entire retrieval architecture secure.
Follow data into logs, exports and support workflows
A protected chat interface does not establish that diagnostic records are equally protected. Identify which request and response fields enter logs, analytics, incident tickets or support exports. Inspect who can access them and whether the permitted support purpose requires that content. Use synthetic fixtures to test the flow wherever possible.
Distinguish evidence capture from unnecessary replication. A security finding may need a marker and linked record identifiers rather than a full copy of every response. If a real exposure occurs unexpectedly, apply the agreed handling process and minimise further access. The report should preserve enough proof to support action without becoming another uncontrolled repository of restricted data.
Review the effect of redaction on test interpretation. If logs remove protected fields, their absence from the exported trace may not establish that retrieval never occurred. Identify the stage at which redaction happens and use other authorised evidence where needed. Privacy-preserving logs can be appropriate, but the resulting audit limitation must remain clear.
Use controlled negative tests with meaningful positive controls
A denied request is informative only if the test conditions are understood. First confirm that the authorised identity can access its permitted synthetic fixture through the intended path. Then test the prohibited identity-operation combination. Otherwise, a universal outage or broken index may look like successful access control because every request fails.
Preserve both positive and negative results. If the control blocks all users, it may prevent disclosure while making the service unusable. If one authorised path works and the prohibited path is denied at the correct boundary, the evidence is stronger for that case. Record technical failures separately from deliberate policy denials.
Repeat relevant tests after material configuration changes, but do not equate a fixed case set with exhaustive security coverage. New tools, roles, data sources and exports may create new boundaries. A risk-based extension of the test matrix should explain what changed and why the selected cases address it.
Interpret evidence in stages
Use precise language for attempted requests, internal access and user-visible disclosure. A submitted request is not a successful breach. An unauthorised passage entering generation context is a different observation from that passage being displayed to a user. Both can matter, but their consequences, containment and evidence requirements differ.
| Observation | Supported conclusion | Unsupported extension |
|---|---|---|
| Prohibited request rejected before retrieval | The tested enforcement point denied this request. | All possible requests and paths are secure. |
| Restricted synthetic text entered model context | The intended upstream isolation failed in this case. | A real client secret was necessarily disclosed. |
| Synthetic marker appeared in another tenant's response | A cross-tenant output boundary failed under the recorded conditions. | The frequency or full historical extent is already known. |
| Tool attempt recorded without destination evidence | An operation was attempted. | The external effect succeeded or failed conclusively. |
| No issue observed in selected tests | No prohibited effect was observed in those cases and conditions. | The application has no security vulnerabilities. |
This ladder helps management act without exaggeration. A confirmed synthetic cross-tenant disclosure can justify urgent containment even when its production prevalence is unknown. The severity guide considers consequence, exposure and uncertainty; a low number of observed cases should not automatically make an authority failure low priority.
Write a finding that identifies the enforceable boundary
Suppose the controlled cache test returns A1's synthetic status marker to Tenant B. The finding should identify the tested application and cache configuration, identities, fixture permissions, ordered requests and observed response. It should state that the boundary failed in this path. It should not claim that all tenants' data was exposed or that a specific historical incident occurred without supporting evidence.
The cause may be an insufficient cache scope, an identity propagation error or a different mechanism. Preserve the cause as a hypothesis until traces or implementation review support it. A prompt change that discourages disclosure may reduce visible symptoms without correcting the shared cache. Assign remediation to the component responsible for enforcing the boundary.
Containment and root-cause correction can be separate decisions. The owner may disable the affected cache path while investigating, provided the resulting service behaviour is understood. Record who authorised the change and whether other routes remain exposed. The audit report should make those operational decisions visible without pretending the auditor controls the production system.
Retest the correction and adjacent permitted behaviour
Retest the original failing sequence with the corrected configuration and fresh controlled fixtures. Examine relevant variations, such as another role or export path, if they share the same mechanism. Confirm that authorised access still works. A correction that denies every request is not equivalent to an effective, usable authorisation control.
Use retesting and closure evidence to identify exactly what is resolved. Retain the original finding, corrective version, new observations and remaining limits. If production history was not assessed, a successful current retest does not answer how many historical records were affected. That may require a separately scoped investigation.
Keep security testing distinct from a broader AI red teaming exercise. Both can contribute evidence, but they may have different objectives, methods and coverage. An audit conclusion should describe the authorised tests actually performed rather than adopting a broad label that implies unrestricted adversarial coverage.
Assess how the test environment supports the production conclusion
A sandbox result is strongest when the relevant control matches production. Compare identity mapping, role policy, retrieval configuration, cache behaviour, tool permissions and logging at the boundary being tested. The environments need not be identical in every detail, but differences that affect enforcement must be identified. A simplified test account with special privileges may produce a result that does not represent an ordinary user.
Document which production configuration evidence can be inspected without running disruptive tests. A versioned policy export, deployment record or witnessed configuration review can help connect the sandbox mechanism to the live service. Such evidence supports configuration correspondence; it does not automatically prove live operation over a historical period. Keep design, configuration and operating evidence distinct in the conclusion.
Shared components deserve explicit attention. A central cache, search service or support platform may serve several applications, while the engagement covers only one assistant. A finding in that component can have wider implications, but the auditor should identify the potential dependency and recommend an owner-led impact assessment rather than declare every connected service affected. The same restraint applies when a shared control passes selected tests.
Preserve a defensible test record
The record should make the controlled sequence understandable to another competent reviewer. Include the fixture permission snapshot, relevant configuration, test identities, ordered actions and evidence references. Explain time differences between systems if they affect the sequence. Do not infer order solely from timestamps whose clocks were not reconciled; request identifiers and destination records may provide stronger linkage.
Retain failed or inconclusive runs as well as successful demonstrations. A missing trace may limit the conclusion without proving that a control failed. A test harness error may invalidate a run and require repetition. State those dispositions transparently. Removing inconvenient runs without a reason can make the security evidence appear stronger than the actual test programme.
At completion, account for test fixtures and access granted for the engagement. Confirm cleanup through the authorised owner and preserve only the evidence required by the agreed retention arrangement. Cleanup does not erase the finding or its audit trail. It closes the temporary testing arrangement while leaving a reproducible record of the conditions, observations and remaining questions.
Prepare a useful scoping pack
Provide the data-flow map, identity and role definitions, permitted operations, test environment options, synthetic fixture plan and available trace evidence. Identify connected suppliers and any limits on testing their services. A clear scope makes the proposed work reviewable and avoids confusing a commercial AI assessment with permission to test unrelated systems.
Discuss authorised testing of your AI security boundaries to agree the protected assets, enforcement points, evidence and stopping rules. The resulting conclusion should make clear what was tested, what was observed and which decisions remain for the system owner.
FAQ
Your questions, answered.
What is an AI security boundary test?
It is an authorised examination of whether a defined limit between identities, data, tools or effects is enforced. A test may compare permitted and prohibited identity-operation combinations using synthetic records. The evidence should show where access was allowed or denied and what effect occurred. A model’s refusal is only one observation; the audit must distinguish conversational behaviour from retrieval, execution and destination controls.
Does a chatbot refusal prove that protected data was not accessed?
No. Restricted material may have entered the model’s context before the final response refused disclosure. Conversely, an inappropriate statement can occur without any protected data access. Inspect authorised intermediate evidence where available and separate attempted requests, internal retrieval and user-visible disclosure. If only final outputs are accessible, qualify the conclusion about upstream controls instead of treating a clean response as proof of complete isolation.
How can cross-tenant AI access be tested safely?
Use approved synthetic tenants, test identities and distinct non-sensitive markers with a documented permission matrix. Confirm the authorised path first, then test the prohibited combination within the rules of engagement. Preserve minimal linked evidence and stop if unexpected exposure occurs. Do not use real client secrets simply to make the demonstration more convincing. Report environmental differences and the exact paths tested rather than claiming universal isolation.
Can cached AI answers bypass the intended access boundary?
They can create a separate path that needs examination. Test whether cache scope reflects the identity, tenant, role and other conditions relevant to the answer. A controlled sequence can populate a cache under one authorised identity and then query under another. The audit should inspect actual behaviour and configuration, including role changes, rather than assume that a question-based cache key either is or is not used.
What permission is required before testing a connected AI supplier?
The rules of engagement must identify which environments, accounts, operations and connected services are authorised. Permission to review the customer-facing assistant does not automatically extend to a supplier’s infrastructure or other tenants. Resolve those boundaries with the system owner and relevant provider before testing. Where access is restricted, use approved alternative evidence and state the limitation rather than expanding the test informally.
How should a confirmed boundary failure be reported?
State the intended rule, tested identities, configuration, ordered steps at a high level and observed effect, supported by controlled evidence. Separate the confirmed condition from a proposed cause and from unknown historical prevalence. Explain consequence and exposure without exaggerating the data affected. A synthetic disclosure can justify urgent action, but it does not establish that real client records were historically exposed unless separate evidence supports that conclusion.
Do AI security tests require retaining full prompts and client documents?
Not necessarily. The evidence should be sufficient for the tested claim and handled under explicit authority. Synthetic fixtures, markers, configuration identifiers and linked records may establish a boundary failure without copying sensitive source material. If logs redact content, understand where redaction occurs before interpreting absence as proof of no access. Preserve audit usefulness while minimising unnecessary replication and controlling access to the evidence.
What proves that an AI access-control fix worked?
Retest the original failing path under the corrected configuration, add relevant variations that share the mechanism and confirm permitted access still works. Retain destination or intermediate evidence as required by the claim. A rejection message alone may be insufficient, and denying everyone is not a usable authorisation design. State remaining coverage and historical-investigation limits separately from verified correction of the current defect.