AgentInjectionBench
Catch unsafe tool calls before you ship.
Your user asks for research. A search result tells your agent to send its private context to an external URL. A model upgrade, prompt edit or new tool changes the agent you already tested, so the check has to be repeatable.
- 182 public synthetic cases: 142 attacks and 40 benign controls across 7 attack categories, each with its task, system prompt, tool definitions and policy checks already assembled.
- Evidence you can inspect: per-case tool calls and arguments in traces.jsonl, with a dataset fingerprint so before-and-after runs compare the same cases.
- Runs against OpenAI, Anthropic, any OpenAI-compatible endpoint such as Ollama, or your own agent through an adapter.
Know the limit
A prepared case pack and grader to run beside garak, Promptfoo, PyRIT or Giskard, not a replacement for them. Provider billing applies when you run it against a paid model.
$ git clone https://github.com/ppradyoth/AgentInjectionBench.git && cd AgentInjectionBench
$ pip install -e '.[openai]'
$ aib-run --adapter adapters.openai_compatible:adapter --limit 10 --seed 42 --bundle results/model-check