Controlled puzzles expose limits in reasoning-model scaling
Apple researchers evaluated reasoning and standard language models on configurable planning puzzles and found three performance regimes. Reasoning helped at medium complexity, but both model types failed at high complexity, with reasoning-token use eventually declining despite available budget.
Why it matters: Longer visible reasoning is not reliable evidence of correct planning, especially as task complexity moves beyond benchmark-like conditions.