Securing Multi-Tenant RAG: Recall Collapse, pgvector Filtering, and PostgreSQL ltree

Every web engineer knows the standard access-control pattern: a request arrives, you verify user permissions, then you query the database. Authorize first, then fetch.

In Retrieval-Augmented Generation (RAG), access control is often discussed in vague, binary terms. In practice, filtering happens at three distinct architectural levels, each with very different security and information retrieval tradeoffs.

---

The Three Tiers of Filtering in RAG

When securing a multi-tenant RAG pipeline, teams typically implement one of three filtering patterns:

Tier 1: Soft / Prompt-Level Filtering

Unfiltered chunks are retrieved and dumped into the prompt, accompanied by an instruction like: “Only answer using documents the user has clearance to view.”

As documented in the OWASP Top 10 for LLM Applications (LLM01: Prompt Injection & LLM06: Sensitive Information Disclosure), system prompts are probabilistic suggestions, not security boundaries. If restricted tokens enter the context window, prompt injection or stochastic synthesis can reconstruct or summarize the data regardless of output guardrails.

Tier 2: In-Memory Application Post-Filtering

The application runs a global vector query for top_k = N, receives the results, and filters out unauthorized chunks in Node.js or Python memory before sending the remainder to the LLM.

From a strict security perspective, in-memory filtering is safe—restricted tokens are discarded before prompt assembly and never reach the model. However, its fatal flaw is Recall Collapse (Context Starvation):

If an employee with restricted clearance asks a question where the 5 most similar embeddings belong to confidential documents, the in-memory filter discards all 5. The LLM receives 0 context chunks, even if dozens of valid, highly-relevant documents exist at ranks 6 through 50.

Tier 3: Query-Level Pre-Filtering

Authorization permissions are compiled directly into the vector database query (WHERE dc.path <@ ...). The database evaluates access rules during retrieval, guaranteeing that the returned candidate set is dense with permitted documents.

---

The pgvector Nuance: Query-Level vs. True Pre-Filtering

Moving the filter into a SQL WHERE clause guarantees that only authorized rows leave PostgreSQL. However, there is an important database engine nuance: pgvector with HNSW does not perform true single-stage graph pre-filtering by default.

Dedicated vector engines like Qdrant (Filtered HNSW) or Weaviate (ACORN inverted allow-lists) evaluate metadata in-place during graph traversal. In contrast, standard pgvector HNSW navigates the vector graph first and checks the WHERE clause internally against table rows:

SELECT dc.chunk_id, dc.content, dc.embedding <=> :embedding AS distance
FROM document_chunks dc
WHERE dc.org_id = :orgId
  AND dc.path <@ :authorizedPath::ltree
ORDER BY distance ASC
LIMIT 5;

The High-Selectivity Trap

If a user has access to only 2% of an organization’s documents, standard HNSW graph exploration will repeatedly land on non-matching nodes, exhaust its search budget (hnsw.ef_search), and return fewer results than the requested LIMIT.

To mitigate this in PostgreSQL, you have three production strategies:

1. Iterative Index Scans (pgvector 0.8.0+): Enable iterative scanning so Postgres automatically expands the graph search until the limit is satisfied:

   SET hnsw.iterative_scan = 'relaxed_order';

2. Physical Table Partitioning: Partition the document_chunks table by org_id or tenant. This isolates vector indexes physically per tenant, converting the query into an exact partition scan.
3. Partial Indexes: For predictable, high-frequency permission boundaries, construct partial HNSW indexes:

   CREATE INDEX idx_public_chunks ON document_chunks 
   USING hnsw (embedding vector_cosine_ops)
   WHERE is_public = true;

---

Implementing Hierarchical RBAC with PostgreSQL ltree

In our implementation, documents exist in an organizational hierarchy (e.g., org_enterprise.dept_engineering.team_backend). We use PostgreSQL’s native ltree extension to model this hierarchy directly in the database.

1. The Schema

Role assignments define whether permissions apply strictly to a node or inherit down the subtree:

CREATE EXTENSION IF NOT EXISTS ltree;

CREATE TABLE role_assignments (
  org_id UUID NOT NULL,
  user_id UUID NOT NULL,
  role_id UUID NOT NULL,
  path ltree NOT NULL,
  should_inherit BOOLEAN DEFAULT true,
  expires_at TIMESTAMPTZ
);

CREATE TABLE document_chunks (
  chunk_id UUID PRIMARY KEY,
  org_id UUID NOT NULL,
  path ltree NOT NULL,
  content TEXT NOT NULL,
  embedding vector(1536) NOT NULL
);

CREATE INDEX idx_chunks_path ON document_chunks USING gist(path);

2. Path Resolution & Filter Compilation

At request time, we resolve active role assignments, prune redundant descendant grants, and compile exact and inherited paths into SQL:

interface AuthorizedPaths {
  exact: string[];
  inherited: string[];
}

export function buildLtreePathFilter(paths: AuthorizedPaths) {
  const conditions: string[] = [];
  const replacements: Record<string, string> = {};

  if (paths.exact.length > 0) {
    const placeholders = paths.exact.map((p, i) => {
      const key = `exactPath${i}`;
      replacements[key] = p;
      return `:${key}`;
    });
    conditions.push(`dc.path IN (${placeholders.join(',')})`);
  }

  paths.inherited.forEach((p, i) => {
    const key = `inheritedPath${i}`;
    replacements[key] = p;
    conditions.push(`dc.path <@ :${key}::ltree`);
  });

  return {
    sql: conditions.length > 0 ? `(${conditions.join(' OR ')})` : 'FALSE',
    replacements
  };
}

• IN matches exact node assignments (should_inherit = false).
• <@ is PostgreSQL’s ltree ancestor operator (org_a.dept_b.team_c <@ org_a.dept_b evaluates to true).

---

Benchmark: ltree vs. String Prefix Matching

We benchmarked three hierarchy encodings across 115,301 sentence chunks (16,701 ArXiv abstracts) on PostgreSQL 17 to measure filtering behavior as hierarchy depth increases:

---

Practical Engineering Considerations

The UUID Sanitation Requirement

PostgreSQL ltree labels only permit [a-zA-Z0-9_]. Standard UUIDs contain hyphens, which cause syntax errors. Sanitize UUIDs into 32-character hex strings on write and reinsert hyphens on read:

export function toLtree(uuid: string): string {
  return uuid.replace(/-/g, '');
}

export function fromLtree(sanitized: string): string {
  return [
    sanitized.slice(0, 8), sanitized.slice(8, 12),
    sanitized.slice(12, 16), sanitized.slice(16, 20), sanitized.slice(20)
  ].join('-');
}

Architecture Trade-Offs

• Dynamic Resolution over Session Caching: Resolving paths per query provides immediate revocation when permissions change, but adds a fast query on role_assignments. For large deployments, cache resolved path sets in Redis with a short TTL.
• Passive Documents: Documents store canonical paths rather than explicit access lists. This centralizes access control in role assignments and eliminates reverse-lookup maintenance when permissions change.

---

Summary

Securing RAG is not just about keeping restricted tokens away from the LLM—in-memory filtering already achieves that. The real engineering objective is preventing recall collapse while maintaining low query latency.

By compiling hierarchical access scopes into database-level queries (ltree) and leveraging engine-appropriate scanning (hnsw.iterative_scan or table partitioning), you ensure both airtight isolation and high context density for every query.