What changed

GitHub released ReviewBench, a versioned benchmark for code-review agents built from 219 public pull requests across 187 repositories and 19 languages. It includes a multisource golden set, matcher, judge and reproducible runner.

Why it matters

Code-review systems can produce plausible comments while missing consequential defects or flooding teams with noise. A public task, matcher and judge make precision, recall and benchmark drift more inspectable.

What to do

Run the public benchmark at a pinned version, inspect false positives and missed findings by language, then add accepted and rejected reviews from your own repositories before rollout.

Caveat

GitHub selected the pull requests and reports the comparative results. A Claude Sonnet 5 grader reached high audited agreement, but model grading and public repositories remain imperfect proxies.