What changed
Sakana AI and University of British Columbia researchers released a coding agent that modifies its own Python code, evaluates descendants and preserves a diverse archive of prior variants. Reported SWE-bench performance rose from 20 percent to 50 percent during the research run.
Why it matters
Agent scaffolds can become objects of automated search, making evaluation design and containment as important as the underlying model.
What to do
Experiment only in an isolated sandbox with fixed budgets, immutable external tests, reviewed patches and no production credentials or network authority.
Caveat
The benchmark gains came from a compute-intensive research setup and do not show safe, open-ended improvement in real production environments.