Data Leakage Risks in Retrieval-Augmented Generation Systems

Data leakage in retrieval-augmented generation systems spans the chain. Sensitive data can leak during retrieval, context construction, or generation, or when access controls fail. A secure RAG deployment must protect every stage, because a strong control at one link cannot make up for a missing one at another.

Why RAG systems create a wider data leakage surface than traditional applications

RAG has become the dominant pattern for enterprise LLM deployments because it supplies current information, reduces hallucination, and makes proprietary data useful without fine-tuning. Those benefits depend on giving the model access to outside knowledge. In a conventional web application, a leak typically starts at a misconfigured endpoint or query. In RAG, the path from stored document to generated answer crosses several independently exploitable stages.

Existing application controls still have a role, but they do not cover that whole path. An authenticated user can get documents they are not allowed to read, and a document they are allowed to read can contain hostile instructions. LeakDojo found that stronger instruction-following correlates with higher leakage risk and that improvements in RAG faithfulness can increase exposure. The faithfulness paradox is blunt. A model that follows retrieved material more closely can also follow extraction commands hidden inside it more closely.

How poisoned knowledge bases turn the retrieval stage into an entry point

An attacker can shape what a RAG system does before anyone asks a question. Attackers enter through the knowledge base. A Survey of Secure Retrieval-Augmented Generation from the Hong Kong Polytechnic University and the Hong Kong University of Science and Technology (Guangzhou) labels this S1, or knowledge poisoning, in its SLOT taxonomy. An attacker inserts crafted documents into the vector store so that their embeddings sit near likely target queries. The retriever then selects those documents as relevant context.

PoisonedRAG, presented by Zou, Geng et al. at USENIX Security 2025, demonstrated a 99% success rate with only five injected documents in a million-document corpus. Retrieval manipulation targets vector similarity much as SEO manipulation targets keyword relevance. A related denial-of-service approach fills the context window with high-similarity, low-quality documents, which push legitimate material out without causing an obvious error.

The poison can also sit dormant until a matching query arrives. A clean response does not prove the store is clean, because the system may not have retrieved the dangerous document yet. Documents should be inspected before indexing, with evidence of their origin and integrity preserved.

How indirect prompt injection travels through retrieved content to reach the model

Indirect prompt injection turns retrieved content into an instruction channel. A web page, email, PDF, or calendar invite can hold commands that the model follows as instructions. The trust boundary breaks at context construction. The model cannot reliably tell legitimate document content apart from adversarial instructions hidden in it, so a retrieval compromise can lead it to act during generation.

OWASP ranks prompt injection as its top LLM vulnerability. EchoLeak, CVE-2025-32711, showed the production stakes when it was disclosed in June 2025 with a CVSS score of 9.3. A hidden instruction in an email caused an enterprise copilot to exfiltrate user data with zero clicks and no user awareness. GitHub Copilot CVE-2025-53773 exposed a different consequence. Malicious content could manipulate Copilot into modifying .vscode/settings.json to enable “YOLO mode,” bypassing user approval and allowing arbitrary shell commands.

These cases do not depend on a user deliberately requesting a leak. Research on Prompt-Injected Data Extraction found that instruction-tuned models can be made to copy their contexts, and that stronger instruction-following makes them more vulnerable. Retrieved-document isolation handles this handoff by sanitizing content before the primary model sees it, without assuming that content is safe because it is relevant to retrieval.

Why access control failures at the retrieval layer are the most common leakage path for internal users

The most common internal leakage path needs no malicious document. A legitimate user asks a routine question, and the retriever returns material that user was never allowed to read. Source permissions do not transfer automatically. When documents from SharePoint, Drive, or Confluence are chunked, embedded, and indexed, their entitlement rules must be preserved deliberately. Otherwise, the model gets restricted text with no record of who is allowed to read it.

NIST SP 800-53 Rev 5 AC-6, Least Privilege, maps directly to this failure because retrieval access exceeds the user’s entitlement. Shared knowledge bases make the consequences broader when teams or tenants have different permissions but retrieval does not enforce those boundaries. The banking case illustrates the operational cost. During synchronization lag between SharePoint, Workday, and the vector database, revoked users could still extract confidential figures through the LLM interface. Maintaining real-time permission parity required four full-time infrastructure engineers writing custom reconciliation cron jobs, and the CISO ordered the cluster shut down.

UpGuard’s 2025 survey found that roughly one-third of accessible Chroma databases were exposing data to anonymous access, with most of those containing more than one collection, indicating production use. Chroma’s default configuration ships with no authentication. Permission-aware retrieval from ingestion onward avoids an attempt to remove unauthorized information after the model has already used it.

How vector database vulnerabilities turn the infrastructure layer into a direct exfiltration path

The vector database is both storage and a service. Its APIs, authentication logic, and dependencies create an attack surface beneath the retrieval rules. ChromaToast, CVE-2026-45829, was published May 18, 2026, with a CVSS score of 10.0 and no patch. The ChromaDB Python FastAPI server acts on client-supplied model identifiers before authentication, including loading code from external HuggingFace repositories. One unauthenticated HTTP request can trigger attacker-controlled code execution with access to environment variables, mounted secrets, API keys, and files readable by the server process.

Milvus CVE-2025-64513 exposes a separate authentication failure. Its AuthenticationInterceptor base64-decodes the incoming sourceID HTTP header and compares it with a hardcoded constant visible in the open-source repository. An attacker who knows that constant can forge the header. Forging the header grants unauthenticated administrative access to read, modify, and delete all data.

LangChain CVE-2025-68664 connects the integration layer to credential theft. The dumps() and dumpd() functions failed to escape dictionary data that a user controlled when it contained the lc key, so later deserialization could treat that data as a legitimate LangChain object. With secrets_from_env=True as the previous default, a payload could extract targeted environment variables, including vector database connection strings, API keys, and cloud credentials. The chain runs from prompt injection through LLM output into downstream deserialization. Neither the model nor the knowledge base needs its own flaw for that integration failure to expose the database.

Why embedding inversion means a vector database breach is a document breach

Embeddings are not a safe substitute for protecting source text. Although widely treated as one-way transformations, they can be partially or substantially inverted to reconstruct the input. Research published in 2024–2025 demonstrates that risk. OWASP’s LLM08:2025 formally recognizes embedding inversion, undermining the assumption that stolen vectors reveal little of value. For threat modeling, a vector database breach must be treated as a document breach, even when reconstruction is incomplete.

That distinction is especially important for regulated information. Patient records, transaction narratives, and privileged communications do not lose their sensitivity when converted into vectors. The fintech reconstruction case illustrates the consequence, with attackers reverse-engineering embeddings into original client investment portfolios at scale. Even with database perimeter controls in place, the contents can cause harm if an attacker gets through.

What output-stage leakage looks like when every upstream control holds

A user may be allowed to read two documents separately without being allowed to learn what their combination reveals. RAG makes that boundary difficult to enforce. A synthesized answer can expose a relationship that neither document states alone, even when both retrieval decisions were valid. The model has no built-in understanding of that distinction. This aggregation problem leaves a residual leakage surface after knowledge base integrity, retrieval permissions, and infrastructure security are addressed.

A shared knowledge base creates further risk when the context contains material with different access entitlements. The model can combine restricted and unrestricted fragments into one fluent response, making the boundary difficult to recognize at the output. The SLOT survey identifies S4 as the output-facing surface for knowledge extraction and privacy attacks. Its documented stage mismatch, however, concerns S1 corpus attacks outpacing L1 defenses.

Output validation provides a compensating control before an answer reaches the user. PII detection, confidential document markers, and semantic similarity checks against known sensitive content can catch disclosures that earlier controls miss. Aggregation needs its own evaluation, because individually permissible fragments may form an impermissible answer. This is where pipeline-wide defense must extend beyond deciding which documents a user can retrieve.

A defense-in-depth architecture that covers the full RAG pipeline

The practical response is to match controls to each attack surface. The SLOT taxonomy pairs knowledge poisoning with integrity and provenance, retrieval-time access manipulation with retrieval hardening, post-retrieval exploitation with isolation, and knowledge exfiltration with access control and confidentiality. Each pairing closes a different gap. Infrastructure hardening sits beneath all four, because an attacker who bypasses database authentication does not need to defeat the application’s retrieval policy.

At ingestion, teams should scan documents for embedded prompt injection attempts before indexing. Checksums and digital signatures can reveal unauthorized modifications, while provenance records make poisoned content easier to identify and remove. Permission data belongs here too. Access rules for each document must go into the retrieval system with its content, including its embeddings and vector-store records, so those rules are in place before any later filtering.

At retrieval, query-time metadata filtering in Pinecone, Weaviate, or Qdrant can restrict results to what a user identity may access. Retrieval logs should also be audited for broad or unusual queries across departments, which can signal misconfiguration or attack. Context construction needs a separate boundary. Sanitization should strip suspicious patterns and limit retrieved content volume, while higher-risk deployments can use a separate LLM call to summarize or validate documents before they reach the primary model.

At output, responses should be checked for PII, confidential markers, and similarity to known sensitive content. The checks must also look at what fragments reveal when they are combined. None of these logical controls replaces network segmentation, authentication enforcement, or a security-critical dependency patching cadence. The BITS-Pilani / Melbourne survey identifies homomorphic encryption and secure multi-party computation as research directions for protecting embeddings at rest and in transit, but deployment-ready implementations remain limited. Those directions are not a substitute for controls available now.

Where compliance frameworks currently fall short on RAG-specific testing requirements

SOC 2, HIPAA, and ISO 27001 were not designed around RAG’s full attack surface. An application-only penetration test can therefore leave the knowledge pipeline unexamined. ISO/IEC 42001:2023 and the NIST AI Risk Management Framework reference adversarial testing as an AI control objective, but the specific requirements are not yet prescriptive. SOC 2 and HIPAA assessors increasingly expect AI-specific testing evidence when AI tools touch customer or patient data. That expectation does not itself define complete coverage.

SOC 2 sets no penetration-testing interval. Its CC4.1 point of focus lists penetration testing among the ongoing and separate evaluations that management may use, and the organization decides what they cover. HIPAA’s current Security Rule requires risk analysis and periodic evaluation. The HHS December 2024 proposed rule added annual penetration testing, but it had not replaced the current rule as of 2026. Neither position makes a conventional audit proof that embedded patient data is protected.

Enterprise procurement in fintech, insurance, and healthcare is beginning to demand AI security attestations in addition to traditional certifications. For teams responding to those demands, the testing scope must include ingestion, database authentication and dependency versions, retrieval permissions, context construction, and output validation. A whitebox approach gives testers access to source code, cloud configurations, and pipeline architecture so that those boundaries can be examined. The evidence needs to show where unauthorized data is stopped.