Secure RAG and AI assistants: a 2026 security baseline

Reference architecture

1. Preserve source permissions end to end

The most important RAG control is simple in principle: if a user cannot access a document in the source system, the assistant should not surface it through retrieval. Avoid copying large knowledge bases into a flat vector index without preserving access-control context. Retrieval should evaluate user identity, source permissions and tenant boundaries at query time or through an equally robust control design.

2. Treat documents as hostile input

A retrieved document can contain instructions such as “ignore previous rules” or hidden content designed to manipulate the model. OWASP continues to identify prompt injection as a core generative-AI security problem. RAG and fine-tuning do not remove that class of risk. Systems should distinguish system instructions from user input and retrieved content, constrain what retrieved text can influence, and avoid granting model output direct authority over consequential actions.

3. Separate knowledge access from action authority

An assistant that can read internal documents is one risk profile; an agent that can send email, update records, create payments or change infrastructure is another. Tool permissions should be explicit, scoped and auditable. High-consequence actions should use deterministic validation and, where appropriate, human approval rather than model confidence alone.

4. Protect embeddings and intermediate data

Embeddings, vector stores, cached prompts and tool traces can all reveal sensitive business information. Apply the same data-governance questions you would apply to primary records: ownership, classification, encryption, tenancy, location, retention, deletion and access logging.

5. Make answers verifiable

For internal knowledge use, source attribution is a security and quality control, not just a usability feature. Users should be able to inspect which documents supported a material answer, whether those documents are current, and whether the assistant is extrapolating beyond them.

6. Build evaluations around failure modes

• Direct and indirect prompt injection.
• Cross-user and cross-tenant retrieval leakage.
• Sensitive-information disclosure in outputs.
• Retrieval of superseded or unapproved documents.
• Incorrect source attribution.
• Tool misuse and excessive agency.
• Hallucinated policy or compliance claims.
• Adversarial files, URLs and embedded instructions.
• Model/version changes that alter behaviour.
• Failure to refuse when evidence is insufficient.

7. Log the events that matter

At minimum, retain enough evidence to reconstruct material incidents: authenticated user, time, assistant/model version, retrieved sources, relevant policy decisions, tool calls, approvals and administrative changes. Privacy and proportionality still matter; logging everything forever is not a substitute for a deliberate audit model.

A practical deployment gate

A system that fails a material gate should remain a bounded pilot until the control is fixed or the risk is explicitly accepted by the appropriate owner.