The risk
No prompt wording reliably stops injection. A document can contain text that asks the model to change its job, use a tool or expose information. A label around that document helps explain its source, but it does not establish a permission boundary.
What happened in production
The case shows why the tools available while reading outside content matter more than a request to ignore hostile instructions.
The fix
- List every path for outside text: web pages, uploads, emails, tickets, tool results and content written by other users. Include paths that enter through another tool rather than directly through chat. Check what reaches the final model context.
- Label and delimit that text as data with its source. Keep your own instructions separate when assembling the prompt. Treat those labels as context for the model, not as enforcement that makes a dangerous tool safe to expose.
- When outside content is in context, remove or gate tools that send data out or change state. Decide what happens if the model follows the document's instructions. Make that outcome harmless through the tool boundary rather than relying on the reply sounding cautious.
- Check document extraction for hidden text before passing it to the model. Inspect zero-width characters, white-on-white text and tiny fonts. Stripping a few characters from plain text does not inspect how text was styled in the source document.
These examples label already-extracted text and expose no action tools in that step. The character filter is only the plain-text part of extraction. Inspect font size and contrast in your own document path before calling this function.
export function documentContext(source: string, extractedText: string) {
const text = extractedText.replace(/[\u200B-\u200D\uFEFF]/g, "");
return {
content: JSON.stringify({ kind: "untrusted_document", source, text }),
availableTools: [],
};
}
import json
def document_context(source: str, extracted_text: str) -> dict:
text = extracted_text.translate(dict.fromkeys(map(ord, "\u200b\u200c\u200d\ufeff")))
return {
"content": json.dumps({"kind": "untrusted_document", "source": source, "text": text}),
"available_tools": [],
}
Your provider adapter must enforce the empty tool list. A field in a local object does nothing if the adapter still sends every tool to the model or executes a tool request from an earlier turn. Check the actual calls your application accepts.
Prove it works
Plant a canary instruction in a test document asking the agent to send a marker to a test destination. Retrieve the document through the same path your users take. Assert that the outbound tool is unavailable or that the execution gate rejects its call.
Inspect recorded tool calls and arguments, not just the final text response. Repeat with a hidden-text fixture from your supported document format and inspect what extraction emits. Keep a benign document in the test too, so you can confirm that reading normal content still works.
Know the limit
Labels and character filters do not identify every hostile instruction. Removing tools limits the actions this step can take, but a later step can reintroduce access. Carry the same untrusted-content assumption across the full request path.
Tools
AgentInjectionBench
Catch unsafe tool calls before you ship.
Prompt Injection CTF
Break a constrained AI, then read the guardrail that stops you.