Result: cognium-dev 4.9.13 reached 91.0% true-positive rate and 17.4% false-positive rate on the full OWASP Benchmark Java v1.2 suite. Its Youden score was 73.5%. This is a dated, reproducible result—not a claim about every future build.
The OWASP Benchmark for Java contains 2,740 test cases: 1,415 are intentionally vulnerable and 1,325 are safe. A useful SAST evaluation must measure both groups. Finding vulnerable cases shows recall; leaving safe cases alone shows selectivity.
Overall cognium-dev result
The September 11, 2026 run used cognium-dev 4.9.13, BenchmarkJava commit 20cbf3d, default scanner configuration, and no LLM. A finding received credit only when it landed in the expected file with the expected CWE after documented normalization.
| Tool | TP | FP | FN | TN | TPR | FPR | Precision | Youden |
|---|---|---|---|---|---|---|---|---|
| cognium-dev 4.9.13 | 1,287 | 231 | 128 | 1,094 | 91.0% | 17.4% | 84.8% | 73.5% |
That means 1,287 vulnerable cases were detected and 1,094 safe cases were left unflagged. The scanner missed 128 vulnerable cases and flagged 231 safe cases. Publishing all four counts matters because a headline score alone cannot show where a tool succeeds or fails.
How the test was scored
- The complete OWASP Benchmark Java v1.2 suite was scanned—no category or case subset.
- The scan used the engine only. No LLM detection or verification layer was involved.
- Every result was evaluated by the same public scorer and expected-CWE rule.
- Raw scans, normalized findings, scorecards, configurations, and false-positive and false-negative case lists were retained.
cognium-dev scan <testcode> -l java -f json
Where the result is strong
Nine of the eleven OWASP categories produced zero false positives in this run. SQL injection reached 93.8% TPR with 0% FPR, while LDAP injection reached 88.9% TPR with 0% FPR. Weak randomness and weak cryptography each reached 100% TPR with 0% FPR.
Where it needs work
XSS is the main weakness in this snapshot. The scanner flagged 192 of 209 safe XSS cases, producing a 91.9% XSS false-positive rate. Those 192 cases account for most of the run's 231 false positives. Command injection also recorded a 31.2% false-positive rate.
Why these numbers differ from the older scorecard
An April snapshot on the main benchmark page contains a historical 1,415-case row. Its case selection and original harness cannot be reconstructed, so it is not used for comparison. The September full-suite result covers all 2,740 cases with a public scorer and is the figure to use when discussing OWASP Benchmark performance.
What to check before trusting any SAST benchmark
- Was the entire dataset scanned, including safe cases?
- Are tool versions, configurations, dataset commits, and scoring rules recorded?
- Can you inspect raw findings and the false-positive and false-negative lists?
- Are limitations and weak categories published alongside strengths?
Inspect the full benchmark.
See the tool comparison, category breakdown, other languages, and links to every public artifact.
Source note: Results were copied from the public September 11, 2026 full-suite evidence currently linked by cognium.dev. If that evidence is superseded, update the article's modified date and results together.