What changed
Anthropic introduced a feature-based comparison method using a dedicated crosscoder to identify concepts that are more represented in one model than another. The research demonstrated the approach across different open-model architectures, including Qwen, Llama and GPT-OSS families.
Why it matters
Aggregate benchmark scores can hide narrow but consequential changes in refusal behavior, domain knowledge, bias or unsafe capability. A high-recall behavioral diff can tell evaluators where to focus expensive testing after a model update.
What to do
Run a model diff before replacing a production checkpoint, cluster discovered features by risk and business relevance, then create targeted prompts and downstream evaluations for the changes that matter.
Caveat
Features are learned interpretations, not ground-truth causes, and cross-model alignment can miss or mislabel behavior. The method is a screening tool, not proof that a model is safe or that a training change caused a feature.