Topics

refreshable benchmarks

1 synchronized story carry this tag.

SABRE releases refreshable stress tests for vision-language models

SABRE turns Markdown test primers into controlled image edits and questions; its 600-image SABRE-Prior set yields only 17.8% to 31.3% macro accuracy across six models.

Why it matters: Regenerable tests can expose visual-prior failures after fixed benchmarks become familiar, while preserving an inspectable task specification and review path.

Read storyOriginal source · arxiv.org