Retrieval with evidence and permissions
RAG Architecture for Saudi Enterprises
Design enterprise RAG around current authorised evidence, bilingual retrieval, document lineage and stage-specific evaluation, with a Saudi engineering-library example.
Retrieval-augmented generation, or RAG, connects a language model to selected source material at answer time. In an enterprise, the difficult part is ensuring that the material is relevant, current and available to this requester. A vector search result is a candidate piece of evidence, not an automatic guarantee that the final answer is true or authorised.
For Saudi organisations working in Arabic and English, the architecture must also preserve identifiers, document meaning and source authority across languages. This guide focuses on the integration pipeline from the document system to a cited answer. It connects to AI integration and delivery without assuming a particular model provider or hosting arrangement.
Start with an answer contract and a source authority rule
Define what a useful answer must contain: a response to the question, the applicable source version, evidence locations and an explicit unresolved outcome when the evidence is insufficient. State what the assistant may not decide. A document-search assistant should not acquire permission to approve engineering changes simply because it can explain a document.
Use an illustrative engineering library. A user asks which approved drawing revision applies to assembly K-42 after change notice ECN-81. The library contains superseded drawings, a future-effective revision and a project-specific amendment. The task is to identify the applicable evidence, not to choose the file with the newest upload timestamp.
Establish an authority rule with the document owner: approved status, effective period, project scope and amendment precedence. If these are missing or contradictory, the assistant should describe the unresolved condition and direct the user to document control. Better generation cannot repair an undefined source-of-truth rule.
Ingest documents with enough lineage to correct them later
Microsoft's RAG design and evaluation guidance treats preparation, retrieval and generation as distinct stages. In this design, ingestion should preserve document identity, source location, revision, approval state, effective dates, language and access metadata. Record the extraction and chunking configuration so an index can be rebuilt deliberately.
Do not strip the structure that establishes meaning. A table heading may define units for every row. A footnote may qualify an inspection interval. A drawing title block may contain the revision that makes the page applicable. Retain those relationships when preparing searchable chunks, even if that requires different handling for prose, tables and drawing metadata.
Version chunks against the source document. When a document is corrected, superseded or deleted, identify which chunks and derived answers depend on it. A reindexing job should not leave old fragments searchable indefinitely because the new file received a different storage name.
Enforce permissions before evidence reaches the model
A user can be signed in and still lack access to a particular project document. Derive project and user scope from trusted application context, then enforce it during retrieval and source access. A prompt instruction such as “do not show confidential files” is not a replacement for this boundary.
Microsoft's secure multitenant RAG guidance describes applying tenant and user access context through the retrieval layer. The same design question arises within one enterprise when departments or projects have different document permissions. Decide whether the implementation uses separate indexes, enforced filters or a combination, and test the actual permission transitions.
Include access removal. A cached answer, queued request or previously retrieved chunk may outlive a permission change. Recheck authority at the appropriate delivery boundary and invalidate derived content according to the data model. Where a source cannot provide reliable access metadata, restrict that corpus until the missing boundary is resolved.
Retrieve exact identifiers and semantic meaning together
Semantic search helps when the wording of a question differs from the document. Exact lexical matching remains important for part numbers, change notices, project identifiers and clause references. For K-42, retrieving a semantically similar K-24 drawing is a failure even if the surrounding paragraphs discuss the same equipment category.
Consider a hybrid retrieval design that combines lexical and vector candidates, then applies the agreed source filters and ranking. Evaluate the candidate set and ranking separately. A reranker can improve ordering, but it cannot recover a required document that was excluded from the candidate set or make an unauthorised document permissible.
Microsoft's retrieval guidance discusses measures such as recall and ranking quality. Use those to diagnose your own workload rather than assuming that a larger top-k is always better. Adding many marginal chunks can increase cost, latency and conflicting context without adding the needed evidence.
Choose the representation and index lifecycle through embeddings and vector databases. Keep the embedding model, dimensions, similarity metric and corpus version compatible. A storage migration or embedding change is a retrieval release that needs evaluation, not merely an infrastructure adjustment.
Make bilingual evaluation reflect how people ask
Test Arabic questions against English sources, English questions against Arabic sources and questions mixing Arabic with identifiers. Preserve K-42 and ECN-81 during query preparation. Evaluate Arabic spelling variants and colloquial phrasing where they appear in the intended audience, but do not normalise away negation, numbers or distinctions that alter the question.
For example, “Which drawing applies after ECN-81?” and “أي مخطط معتمد ينطبق بعد ECN-81؟” should lead to equivalent evidence under equivalent permissions. They need not generate identical wording. A query asking which revision does not apply must not be collapsed into the positive question.
Do not use translation quality as a proxy for retrieval quality. A beautifully translated question may still point to the wrong project. Conversely, retrieval may find the right amendment while generation misstates its effective date. Keep those failures separate so the next engineering change addresses the actual cause.
Generate an answer that can be checked
Pass selected evidence with stable source references and the authority metadata needed to interpret it. Require the answer to distinguish source-supported facts from unresolved interpretation. Link citations to the actual document or permitted evidence view rather than inventing a file title or a page location.
For the drawing example, a useful answer identifies the applicable approved revision, cites the amendment establishing it and explains any effective-date limitation. If two approved documents conflict without a precedence rule, it should show the conflict and stop short of selecting one. “No sufficient authorised evidence found” is different from “no such document exists”.
Treat retrieved content as data, including when it contains instructions aimed at the model. The model should not gain a tool or external-action privilege from a document. Implement the surrounding guardrails and keep action services separate from read-only answering. RAG improves access to evidence; it does not remove prompt-injection risk or make unsupported conclusions impossible.
A worked evaluation protocol
The following numbers are illustrative design calculations, not benchmark results. Build 200 cases: 60 answerable Arabic questions, 60 answerable English questions, 40 answerable mixed-language or identifier-heavy questions, and 40 cases without sufficient authorised evidence. Include superseded revisions, future-effective documents, cross-project similarity and permission removal.
| Layer | Question | Evidence |
|---|---|---|
| Ingestion | Is the applicable source represented correctly? | Extracted text, metadata, tables and chunk lineage |
| Access | Can this requester retrieve this source now? | Allowed and denied cases, including revoked access |
| Retrieval | Was the required evidence among the selected candidates? | Labelled relevant sources and candidate ranking |
| Generation | Does the answer accurately express the evidence? | Claim support, scope, dates and citation correctness |
| Abstention | Does insufficient evidence produce the agreed unresolved response? | Unanswerable and contradictory cases |
| Operations | Is the result useful within the task deadline? | Latency, review effort, failures and source freshness |
If the required evidence is retrieved for 148 of the 160 answerable cases, evidence recall for that defined set is 148 divided by 160, or 92.5%. If 38 of the 40 insufficient-evidence cases receive the correct unresolved response, that separate rate is 95%. Do not combine them into an unexplained overall accuracy score.
Measure answer correctness only with a stated denominator and grading rule, and break results down by language and failure class. An observed zero access failures in a test set does not prove that every possible permission path is safe. Use integration testing to extend the evaluation to the actual application boundary and release process.
Operate freshness, cost and source changes
Set a freshness objective for each corpus. An approved drawing update may require a different propagation deadline from a historical reference collection. Measure source-change-to-searchable time and removal time. Distinguish an ingestion delay from a retrieval miss so the support team does not keep changing prompts to fix an indexing backlog.
Apply AI caching strategies only with source, permission and configuration versions included in the validity decision. A cached answer from yesterday may cite the correct document for yesterday and still be wrong today. Track the cost of useful completed answers, including retrieval, reranking, generation and review.
For a Saudi deployment, document the actual data flows, storage, logging and support access rather than inferring suitability from a language label or region name. Review applicable obligations with the responsible owners through PDPL integration planning. The architecture should make those decisions inspectable without asserting that every organisation has identical requirements.
Trusted Data's RAG architecture guide is useful related commercial reading. Discuss an enterprise RAG integration with a representative question set, source-authority rules and the permission boundaries that a cited answer must preserve.
FAQ
Your questions, answered.
Can one RAG system answer Arabic questions from English documents?
It can be designed to do so, but the capability must be evaluated on the actual corpus and questions. Test cross-language retrieval, exact identifiers, negation and source authority separately from answer fluency. Preserve the original evidence and cite it clearly. A good translation does not establish that the correct project document or revision was retrieved.
Does RAG prevent hallucinations or prompt injection?
No. Retrieval supplies candidate evidence, but the model can still misinterpret it, overstate its support or follow malicious instructions embedded in content. Evaluate retrieval and generation separately, enforce access outside the prompt and keep external actions behind independent authorisation. Use clear unresolved outcomes when evidence is missing or contradictory rather than treating every question as answerable.
How should enterprise RAG handle superseded documents?
Preserve source identity, revision, status and effective period, then apply the agreed applicability rule during retrieval and answering. Superseded material may remain useful for historical questions but should not silently answer a current-state question. Link replacements and amendments, update affected chunks and invalidate derived answers when their evidence is no longer applicable.
Should a Saudi enterprise use a separate vector index for every department?
That is an architecture choice, not a universal requirement. Separate indexes can simplify some isolation boundaries while increasing provisioning and operations. Shared indexes require enforceable access filtering and careful tests. Choose from the actual permission model, scale, source capabilities and responsible data-handling requirements, then test grant, removal and cross-department access paths.
Which metrics reveal why a RAG answer is wrong?
Inspect ingestion completeness, permission filtering, evidence recall, ranking, claim support, citation correctness and the handling of insufficient evidence. State denominators and break results down by language and case type. An overall satisfaction or accuracy percentage alone cannot reveal whether the missing evidence was never indexed, filtered incorrectly, retrieved poorly or interpreted incorrectly.
What should we provide for an enterprise RAG project enquiry?
Provide representative permitted questions, source-system details, document authority rules, language requirements and the current access model. Include examples that should not be answered and documents whose versions conflict. A masked sample corpus is useful, but unrestricted production access is unnecessary for the first discussion. These inputs define the evidence and permission boundaries the architecture must preserve.