What changed
Researchers released ExpertCrowd-SWE, a 100-task coding-agent benchmark across 87 open-source projects and 11 languages. Tasks use expert-authored change requests, sealed reference solutions, partial-credit graders and adversarial control patches; 24 tasks are public and 76 remain held out.
Why it matters
The benchmark targets larger, merge-quality changes rather than short bug fixes and repeats each model five times. Its leading scores around two-thirds show that frontier coding agents still fail substantial portions of realistic unfamiliar-repository work.
What to do
Run the 24 public tasks with a pinned agent harness and budget, compare repeated-run variance and reviewer effort, and keep internal repository acceptance tests as the deployment gate rather than importing leaderboard rank.
Caveat
This is a preprint and benchmark release, not peer-reviewed evidence. Most tasks are private, model runs use one platform harness and the reported ranking may change with tools, prompts, budgets or contamination.