CodeAnt AI benchmark
Benchmark · August 8, 2026
16 seeded patterns. 11 caught. The harder misses tell the story.
We scored a CodeAnt scan against a prewritten answer key, then compared the same FastAPI target with Dependabot, Bandit, Semgrep, and CodeQL. One sink-free refund stub was excluded from scoring. CodeAnt also found unplanted hardening and LLM cost-control issues.
Why it matters. CodeQL caught the path traversal and SSRF patterns that CodeAnt missed. Broken access control, LLM prompt injection, and mass-assignment remained missed across the applicable code analyzers. Different tools covered different parts of the problem.
Inspect the tested repository ↗
Evidence records
The scored benchmark
| Seeded tier | Scored | Detected | Missed |
|---|---|---|---|
| Signature / sink patterns | 11 | 10 | 1 |
| Semantic / logic / AI | 5 | 1 | 4 |
| Total | 16 | 11 | 5 |
August 8, 2026 · BENCHMARK-REPORT.md · Scoreboard; codeant-analysis-report.json
The misses, with a useful control
| Seeded pattern | CodeAnt | CodeQL |
|---|---|---|
| Path traversal | Missed | Detected |
| SSRF via redirect | Missed | Detected |
| Broken access control | Missed | Missed |
| LLM prompt injection | Missed | Missed |
| Mass-assignment | Missed | Missed |
August 2026 · PUBLIC-REPORT.md · Industry comparison; scans/codeql.txt; raw CodeAnt report
Know the limit
Static pattern-level evaluation at commit 63ad392, 323 lines analyzed. The target has no database schema and does not run. These results are historical coverage measurements, not live exploitation or vulnerabilities in CodeAnt.
How to prevent this
No guide controls reference this record yet.
All work here was performed independently, on personal time, equipment, and accounts, outside and unrelated to any employment. It uses no employer systems, data, or resources and does not represent any employer's views. Public records omit attack payloads and sensitive source material. Full reproduction is available to the affected vendor on request.