What changed
MerchantBench runs agents through 365 simulated days of e-commerce operations using 98,843 product records and 26 tools. Across eight models, two frameworks, and 48 runs, the best configuration reached 27.3% of humans' mean final net assets.
Why it matters
Locally reasonable pricing, inventory, supplier, and cash-flow choices can compound into poor long-term outcomes even when individual tool calls appear successful.
What to do
Evaluate operational agents across complete planning cycles with delayed feedback, cash constraints, revisited decisions, and irreversible effects instead of grading isolated actions.
Caveat
This is an author-run simulation, not a live-business deployment. Results depend on its economic rules, human comparison group, and agent harness.