Topics
2 synchronized stories carry this tag.
CARE-Bench evaluates 500 cases across 1,059 patient-disclosure prefixes. Prompted macro-F1 ranged from 46.9 to 63.4, and models preserved needed clarification only 33.5% of the time.
Why it matters: Medical safety depends on asking the right question before recommending action, not merely reaching a plausible final answer.
OpenAI released HealthBench, an open evaluation containing 5,000 realistic health conversations and 48,562 physician-written rubric criteria. The dataset spans multilingual patient and clinician scenarios and was developed with 262 physicians who have practiced across 60 countries.
Why it matters: Medical AI needs evaluations that test context seeking, communication and worst-case behavior rather than exam recall alone.