DataClawEval makes data agents execute real pipelines
DataClawEval provides 100 Docker-isolated tasks across PySpark, MySQL, HiveSQL, Trino and FlinkSQL. Agents inspect schemas, implement pipelines, execute them and materialize artifacts that deterministic scripts grade; the strongest reported configuration scored 74.9.
Why it matters: Text-to-SQL scores omit most of the work required to build and maintain real data systems, including execution, artifacts and engine-specific behavior.