Project Vend tests an agent running a real shop
Anthropic and Andon Labs let Claude Sonnet 3.7 manage an office shop for about a month using search, notes, customer chat and delegated physical labor. It sourced products and resisted some jailbreaks but lost money through weak pricing, discounts, memory failures and hallucinations.
Why it matters: Long-running economic tasks expose compounding errors and state-management failures that short agent benchmarks rarely reveal.