Your RAG search checks permissions. Does neighbour expansion?

A search result passes the caller’s permission filter. The application then fetches the paragraphs around it to give the model enough context. That second fetch uses a parent document ID and a position range, with no permission check.

The search was safe. The assembled prompt contains restricted text.

I would make context assembly an explicit authorisation boundary in a RAG system. Every operation that adds evidence needs an access decision for the evidence it adds. A permission check on the initial hit cannot grant access to arbitrary neighbours, linked records or a reconstructed parent document.

This article demonstrates an application-level failure with synthetic chunks. It is not a report of a vulnerability in Azure AI Search or another vendor’s retrieval service. The example needs neither a model nor a malicious prompt.

Where expansion changes the access question

Retrieving short chunks can produce an awkward answer when a useful definition is in the preceding paragraph. A developer might fetch adjacent chunks, load the parent document, or follow references to improve the context. Each operation increases the set of source material used in the answer.

Microsoft’s document-level access documentation describes security filters that exclude search results without a matching user or group identity. It also distinguishes that string-comparison technique from native identity-based permission features, several of which are marked as preview. Those documented search controls do not establish what an application’s separate document-store fetch authorises.

Suppose search returns chunk a0 from tenant alpha. A later query asks for every chunk with parent="guide" within one position of a0. Two mistakes can now matter:

• Another tenant also uses guide as a parent ID. A lookup without the tenant predicate can select its chunks.
• The application permits chunks under the same logical parent to have different access rules. A neighbouring finance-only chunk cannot inherit the seed’s staff access.

The second case depends on your data model. If every child demonstrably inherits one immutable parent ACL, the access decision can operate on that parent and its verified children. Do not pretend a per-chunk policy exists where the source has only document permissions. Equally, do not assume uniform inheritance when ingestion merges records or the application supports section-level restrictions.

The rule has a familiar API equivalent. OWASP’s guidance on broken object-level authorisation requires checks that the caller can perform the requested action on the requested object. Knowing an object ID, or being allowed to use the endpoint, is insufficient. Applying that principle to an internal expansion helper is my design recommendation here.

Run a seed-only check against a mixed-permission fixture

Download and inspect the standard-library Python example. It contains an intentionally unsafe branch and a guarded branch, plus regression tests:

python3 rag-neighbour-authorisation.py
python3 rag-neighbour-authorisation.py --test
python3 -O rag-neighbour-authorisation.py --test

The caller belongs to tenant alpha and group staff. Every chunk uses parent ID guide:

With seed a0 and radius one, the candidate lookup returns a0, b0 and a1. Chunk a2 is outside the window. The script prints:

seed_only=a0,b0,a1
seed_only_body_reads=a0,b0,a1
per_chunk=a0
per_chunk_body_reads=a0

The unsafe branch checks a0, then accepts every expansion candidate. It reads both the other tenant’s body and the finance-only body. The guarded branch checks the tenant and group predicate on each candidate, so it reads only a0.

The body-read trace matters. Testing only the final answer would miss restricted text sent to a reranker, a model or a debugging trace before a later filter removed it. In a real pipeline, record safe identifiers at those boundaries and test the actual payloads under controlled conditions. Avoid copying confidential bodies into routine logs just to prove they were excluded.

Decide access before loading bodies

The fixture separates candidate selection from body reads. Its core loop is:

for seed_id in seed_ids:
    seed = store.chunks.get(seed_id)
    if seed is None:
        raise ValueError("unknown seed")
    if not policy.allows(principal, seed):
        continue
    for chunk in candidates(store, seed, radius):
        if policy.allows(principal, chunk):
            selected[chunk.id] = chunk

Only after those decisions finish does it call read_body for the selected IDs. A denied neighbour is omitted. A policy exception aborts the operation before any body reads; it does not return an incomplete context whose authorisation status is uncertain.

This is an in-memory teaching model. The trusted fixture already holds all chunk bodies in its process, so its trace proves calls to the body-loading interface, not physical isolation of secret bytes. A production implementation should query trusted metadata or use an authorisation-aware storage operation before fetching bodies into the consumer’s context. If a retrieval API returns text alongside metadata, checking that metadata afterwards cannot undo the fetch.

The Principal object is also supplied by the fixture. In a service, derive the tenant and group membership from authenticated, trusted identity context. Do not accept group names from the question or a client-editable request field. Likewise, source permission metadata needs a trusted ingestion path. The policy’s comparison is useless if the caller can write its inputs.

For performance, the access decision can be a batch operation or an enforced storage predicate. A separate network call per chunk is not the goal. The required property is that each admitted body is covered by a valid decision for this principal and this resource.

Test omissions, outages and useful context

The example’s tests cover more than the visible cross-tenant failure. Missing ACL metadata and an empty allowed-group set both deny access. A denied seed cannot start expansion. An authorised neighbour is retained, so the repair does not simply turn expansion off.

A policy failure before evaluation or partway through candidate checking raises an exception with no body-loading calls. Unknown seed IDs also fail before bodies are read. Repeated seeds produce no duplicate bodies, and duplicate chunk IDs are rejected when the store is constructed. The tests run under normal Python and python3 -O; their checks use unittest rather than removable bare assertions.

These are deterministic regression tests, not evidence of an attack against a deployed service. They omit concurrent policy changes, storage failures and distributed caches. In particular, a revocation between the metadata check and body use remains a production design question. Use a supported atomic authorisation/read operation where available, or define the consistency and revocation guarantees you can enforce. The earlier RAG permission-cache article addresses that separate freshness problem.

Access control also affects answer quality. When a denied neighbour supplied the missing definition, the model may have too little evidence to answer. Keep that outcome visible. Do not recover recall by retrying expansion with a privileged service identity. The top-k filter-order example shows why authorised recall needs its own test, alongside disclosure prevention.

Put the assertion at the last evidence boundary

Inventory every path that can add material after initial retrieval: adjacent chunks, full-parent reconstruction, linked records, reranker inputs and citation previews. For each path, identify the resource whose permission is checked, the identity used and the body-loading call it protects.

Then add one regression case to your own pipeline: an allowed seed beside a denied neighbour containing a unique synthetic marker. Require that marker to be absent from the model payload and any downstream consumer that lacks permission. Include an allowed neighbour too, so the test demonstrates useful expansion rather than a blanket refusal.

For the wider operating model, my Enterprise AI Agent Security book covers RAG governance and delegated identity. Buy the Packt Kindle edition on Amazon UK.