AI assistant exposed restricted document content: diagnosis and fixes

TL;DR — If an AI assistant revealed content from a document a user should not see, the most common root cause is retrieval running without per-user authorization at query time. Start by reproducing the exact request, then inspect retrieval logs for the returned document IDs and verify the app checked ACLs before sending chunks to the model. Reading time: ~6 min

The scenario

It is 3:40 PM on a Tuesday. A customer forwards a screenshot where your in-app AI assistant answers a question with a paragraph copied from an internal HR policy doc they definitely should not have access to. Your app logs show a normal 200 on the chat request, the model provider dashboard shows no errors, and the document itself is not publicly reachable over HTTP. You need to determine whether the leak came from retrieval, stale cache, prompt stuffing, training data contamination, or a bad document ACL sync before legal asks for a blast radius.

Symptoms

• User sees a normal assistant response containing exact or near-exact text from a restricted document.
• Chat/API request returns 200 OK; no application error is raised.
• Retrieval logs show chunks from a document ID outside the caller's allowed scope.
• Access logs for the document service may show no direct GET /document/:id by that user at all.
• Typical app log shapes:

{"level":"info","msg":"chat request","user_id":"u_123","session_id":"s_9f","query":"what is parental leave policy"}
{"level":"info","msg":"retrieval results","user_id":"u_123","doc_ids":["doc_77","doc_12"],"top_k":5}
{"level":"warn","msg":"authorization skipped for retrieval result","doc_id":"doc_77","reason":"service token used"}

• If caching is involved, you may see the same answer served to multiple users with different identities:

$ curl -si https://app.example.com/api/chat/answer?id=ans_4a | sed -n '1,12p'
HTTP/2 200
content-type: application/json
cache-control: public, max-age=300
vary: Accept-Encoding
x-cache: HIT

• If the source was prompt injection from an attached/public document, logs often show tool calls that fetched unrelated docs after processing attacker-controlled instructions.
• If the source was a stale index, the document may have been removed from the app but still present in the vector store or search index.

Likely causes

Step-by-step diagnosis

1. Reproduce with the exact user identity and capture source references.

   curl -sS https://app.example.com/api/chat \
     -H "Authorization: Bearer $USER_TOKEN" \
     -H 'Content-Type: application/json' \
     -d '{"message":"what is parental leave policy?","session_id":"s_9f","debug":true}' | jq .
   

   This is your problem if the response includes sources, doc_ids, or quoted text pointing at a restricted document. If yes, continue to step 2.

2. Check whether retrieval returned unauthorized docs before generation.

   grep -E 'retrieval results|rerank results|authorization skipped' /var/log/app/app.log | tail -n 100
   

   This is your problem if you see the restricted doc_id in retrieval results for the affected user_id, or a line like authorization skipped. Jump to Fixes → Retrieval executed without per-user ACL filtering.

3. Compare app DB ACLs with index/vector metadata for the leaked doc.

   psql "$DATABASE_URL" -x -c "select id, owner_id, visibility, updated_at from documents where id='doc_77';"
   

   Then inspect index metadata using your index API/CLI:

   curl -sS "$VECTOR_API/documents/doc_77" -H "Authorization: Bearer $VECTOR_TOKEN" | jq .
   

   This is your problem if DB says visibility='private' or ownership changed, but index metadata still shows old tenant/user/group access. Jump to Fixes → Document ACL changed in app DB but not in index/vector metadata.

4. Rule out shared caching of answers.

   curl -si https://app.example.com/api/chat/answer?id=ans_4a -H "Authorization: Bearer $USER_A" | sed -n '1,20p'
   curl -si https://app.example.com/api/chat/answer?id=ans_4a -H "Authorization: Bearer $USER_B" | sed -n '1,20p'
   

   This is your problem if you see cache-control: public, missing Vary: Authorization on cacheable GETs, or identical ETag/x-cache: HIT across users. Jump to Fixes → Cached assistant response shared across users.

5. Inspect prompts and tool policies for auth bypass language.

   grep -RniE 'ignore permissions|bypass auth|fetch any document|all documents|service token' ./prompts ./config ./src
   

   This is your problem if the system prompt or tool wrapper instructs the model/tool to use a broad service credential or to search globally. Jump to Fixes → System prompt or tool policy allowed unrestricted fetches.

6. Verify the restricted/deleted doc is gone from search/vector storage.

   curl -sS "$VECTOR_API/search" \
     -H "Authorization: Bearer $VECTOR_TOKEN" \
     -H 'Content-Type: application/json' \
     -d '{"query":"parental leave policy","top_k":10,"filters":{"doc_id":"doc_77"}}' | jq .
   

   This is your problem if doc_77 still returns after restriction/deletion. Jump to Fixes → Deleted/restricted document still present in embeddings/search index.

7. Check for session/user context mix-up.

   grep 's_9f' /var/log/app/app.log | tail -n 50
   

   This is your problem if the same session_id is associated with multiple user_id values or memory rows. Jump to Fixes → Conversation memory/session mix-up returned another user's prior context.

Fixes

Retrieval executed without per-user ACL filtering

Apply authorization before chunks go to the model, not after generation.

// bad: search first, auth later or never
const results = await vector.search({ query, topK: 8 });

// good: enforce tenant/user/group filters in retrieval
const aclFilter = {
  tenant_id: user.tenant_id,
  or: [
    { visibility: 'public' },
    { owner_id: user.id },
    { reader_group_ids: { in: user.group_ids } }
  ]
};
const results = await vector.search({ query, topK: 8, filter: aclFilter });

If your retriever only supports metadata filters, re-index docs with tenant_id, owner_id, and group IDs in metadata. Remove any service-token-only fetch path used on behalf of end users.

Verify it worked:

curl -sS https://app.example.com/api/chat -H "Authorization: Bearer $USER_TOKEN" -H 'Content-Type: application/json' -d '{"message":"what is parental leave policy?","debug":true}' | jq '.sources'

Expected: restricted doc_77 is absent.

Document ACL changed in app DB but not in index/vector metadata

Re-sync metadata for the affected docs, then backfill all recently changed ACLs.

# one-off reindex job for a single doc
node scripts/reindex-document.js --doc-id doc_77 --refresh-metadata

# backfill docs changed in last 7 days
psql "$DATABASE_URL" -At -c "select id from documents where updated_at > now() - interval '7 days';" \
  | xargs -n1 -I{} node scripts/reindex-document.js --doc-id {} --refresh-metadata

If you store ACLs in a separate table, join them into the index payload during indexing rather than assuming the search layer can query your relational DB live.

Verify it worked:

curl -sS "$VECTOR_API/documents/doc_77" -H "Authorization: Bearer $VECTOR_TOKEN" | jq '.metadata'

Expected: metadata reflects current tenant/owner/group ACLs.

Cached assistant response shared across users

Do not publicly cache personalized AI responses. Mark them private or disable intermediary caching.

For nginx:

location /api/chat/ {
    add_header Cache-Control "private, no-store" always;
    add_header Vary "Authorization, Cookie" always;
    proxy_no_cache 1;
    proxy_cache_bypass 1;
}

For app responses:

Cache-Control: private, no-store
Vary: Authorization, Cookie

If you must cache, key by user ID and tenant ID at minimum; do not key only by URL.

Verify it worked:

curl -si https://app.example.com/api/chat/answer?id=ans_4a -H "Authorization: Bearer $USER_A" | grep -Ei 'cache-control|vary|x-cache'

Expected: private, no-store; no shared cache hit.

System prompt or tool policy allowed unrestricted fetches

Remove broad instructions and force tools to require caller context.

{
  "tool": "search_documents",
  "input_schema": {
    "type": "object",
    "properties": {
      "query": {"type":"string"},
      "user_id": {"type":"string"},
      "tenant_id": {"type":"string"}
    },
    "required": ["query", "user_id", "tenant_id"]
  }
}

// tool wrapper must derive auth context server-side
app.post('/tools/search_documents', authn, async (req, res) => {
  const { query } = req.body;
  const { id, tenant_id, group_ids } = req.user;
  const results = await vector.search({ query, filter: { tenant_id, reader_group_ids: { in: group_ids }, or: [{ owner_id: id }, { visibility: 'public' }] } });
  res.json(results);
});

Also delete any prompt text telling the model to "search all knowledge" or use a service credential.

Verify it worked:

grep -RniE 'search all knowledge|ignore permissions|service token' ./prompts ./config ./src

Expected: no matches relevant to runtime behavior.

Deleted/restricted document still present in embeddings/search index

Purge the document from the index and re-embed only after ACL-safe metadata is attached.

⚠️ Deleting from the wrong namespace/index can remove searchable content for other tenants. Confirm tenant/index identifiers before running purge commands.

curl -sS -X DELETE "$VECTOR_API/documents/doc_77" \
  -H "Authorization: Bearer $VECTOR_TOKEN"

# optional: rebuild from source of truth
node scripts/reindex-document.js --doc-id doc_77 --refresh-metadata

If your system uses soft deletes, add a deleted_at filter to every search query and to offline evaluation jobs.

Verify it worked:

curl -sS "$VECTOR_API/search" -H "Authorization: Bearer $VECTOR_TOKEN" -H 'Content-Type: application/json' -d '{"query":"parental leave policy","top_k":10,"filters":{"doc_id":"doc_77"}}' | jq '.matches | length'

Expected: 0.

Conversation memory/session mix-up returned another user's prior context

Invalidate the bad session, fix session ownership checks, and stop loading memory by session_id alone.

-- inspect ownership
select session_id, user_id, tenant_id, updated_at from chat_sessions where session_id = 's_9f';

-- revoke/expire the bad session
update chat_sessions set expired_at = now() where session_id = 's_9f';

// bad
const memory = await db.chat_messages.findMany({ where: { session_id } });

// good
const memory = await db.chat_messages.findMany({ where: { session_id, user_id: req.user.id, tenant_id: req.user.tenant_id } });

Verify it worked:

grep 's_9f' /var/log/app/app.log | grep -Eo 'user_id":"[^"]+' | sort -u

Expected: exactly one user ID for the session.

Prevention

• Add a retrieval authorization audit log and alert on mismatches.

{"event":"retrieval_acl_check","user_id":"u_123","doc_id":"doc_77","allowed":false,"blocked":true}

Alert if allowed=false and doc_id still appears in model context.

• Add CI tests that simulate cross-user retrieval.

npm test -- retrieval-auth.spec.ts

Example assertion: user A querying a term unique to user B's doc returns zero matching sources.

• Pin cache behavior for AI endpoints at the edge and in app code.

location ~ ^/api/(chat|assist) {
    add_header Cache-Control "private, no-store" always;
    add_header Vary "Authorization, Cookie" always;
}

• Run a periodic ACL/index drift job.

psql "$DATABASE_URL" -At -c "select id from documents where updated_at > now() - interval '1 day';" | while read id; do
  app_acl=$(psql "$DATABASE_URL" -At -c "select visibility from documents where id='$id';")
  idx_acl=$(curl -s "$VECTOR_API/documents/$id" -H "Authorization: Bearer $VECTOR_TOKEN" | jq -r '.metadata.visibility')
  [ "$app_acl" = "$idx_acl" ] || echo "ACL drift: $id"
done

• Store document IDs and ACL metadata with every chunk; do not rely on filename/path-only indexing.

{"chunk_id":"ch_991","doc_id":"doc_77","tenant_id":"t_4","owner_id":"u_9","reader_group_ids":["hr"],"deleted_at":null}

• Add a kill switch to disable retrieval augmentation without taking the whole assistant down.

export RAG_ENABLED=false
systemctl restart app

Use it during incident response if you cannot trust retrieval results yet.

This article was written by an AI system and published pending human review. Verify anything you intend to act on.