Benchmark
What we have measured, and what we have not.
We publish only numbers we reproduced ourselves, with the date and the dataset version. Everything else is labelled as published by someone else.
zk-audit, reproduced
Run of 2026-10-23: a harness check, not a detection result.
| Dataset | Bugs | Evaluated | Detected | Unsupported | False positives on fixed |
|---|---|---|---|---|---|
| zkBugs @ 212cc50 | 140 | 0 | 0 | 140 | 0 of 0 |
| Bundled fixtures (harness self-check) | 2 | 2 | 2 | 0 | 0 of 2 |
The harness ran against the full catalogue and marked all 140 bugs unsupported, because its adapter assumed a directory layout the real dataset does not use. That is a harness gap, not a detection result. The two bundled fixtures only show the harness works; they say nothing about detection power.
Next: adopt zkBugs' own runner contract (print_bug_vars.sh), re-run, and report direct and original modes with location-matched true positives. Results post on this page once a run evaluates real cases.
The zkBugs dataset has no Noir entries. A public Noir track is planned.
Detection score
Location-matched zkBugs run: in progress.
A location-matched detection score (recall on the vulnerable commits, false positives on the fix commits) is being run against a pinned zkBugs checkout. No figure is shown until that run has evaluated real cases; the results file it writes is published here unedited.
Published by others, not re-measured
The field, on the Circom bugs.
| Tool | Bugs found of 70 | Source |
|---|---|---|
| circom-auditor (Claude Opus 4.8) | 66 | zkSecurity, zk-skills v0.1.0, Aug 2026 |
| circom-auditor (Codex) | 64 | zkSecurity, zk-skills v0.1.0, Aug 2026 |
| Ecne | 25 | bugs.zksecurity.xyz tools evaluation |
| zkFuzz | 25 | bugs.zksecurity.xyz tools evaluation |
| Civer | 12 | bugs.zksecurity.xyz tools evaluation |
| ConsCS | 11 | bugs.zksecurity.xyz tools evaluation |
| Picus | 10 | bugs.zksecurity.xyz tools evaluation |
| Circomspect | 0 | bugs.zksecurity.xyz tools evaluation |
Free tools such as zkSecurity's circom-auditor and zkhydra exist and score well on this dataset. zk-audit does not claim to beat them on Circom recall. The case for zk-audit is Noir support, findings that are machine-checked and replayable, CI with signed receipts, and post-quantum readiness.
These figures are taken from the cited publications and the research notes of 2026-10-23. Check the sources before quoting them.
Production circuits
False positives on audited code.
In a July 2026 re-check of 97 already-audited production circuits, 89 were analysed and 8 were inconclusive; none produced a false-positive FAIL. This measures false positives only, not detection. It used the Circomspect engine only, predates every engine added since, and has not yet been re-measured.
Recorded 2026-07-10 (zk-audit commit 2fda2b1), Circomspect 0.8.1 engine, circomlib include resolution on. Corpus: circomlib 57 circuits, MACI 37, tornado-core 2, Semaphore 1. The source commits of those four projects were not recorded.