OWASP Benchmark
The OWASP Benchmark is a free, open test suite of thousands of deliberately vulnerable and safe code samples used to measure how accurately a static application security testing tool detects real vulnerabilities while avoiding false positives.
The OWASP Benchmark is an open-source project that provides a large, fully labeled Java (and now Python) test application containing thousands of test cases. Each test case is a small code snippet that is either a genuine vulnerability (a true positive) or a safe look-alike (a true negative), covering categories such as SQL injection, cross-site scripting, path traversal, and weak cryptography.
Because every test case has a known correct answer, a scanning tool's output can be scored objectively. The project computes true positive rate (recall) and false positive rate, then combines them into a Benchmark score, typically visualized on a chart where the diagonal represents random guessing. A higher score reflects a tool that finds real issues without drowning teams in noise.
For developers and buyers, the Benchmark offers a transparent, reproducible way to compare detection accuracy rather than relying on marketing claims. Its main limitation is that synthetic test cases do not fully represent messy production codebases, so it is best treated as one signal among several, alongside testing on real-world vulnerable applications.