APort Vault tests deterministic payment authorization against 4,371 human attacks
APort Vault reports 225,964 evaluations replaying 4,371 human-authored attacks across 14 models, five policy levels and two tracks. The authors observed 140 unpermitted transfers in model-only runs at policy levels 2–4 and zero of 69,297 evaluations when a deterministic pre-action authorization layer was enforced; they released data and scoring code.
Why it matters: The benchmark separates a model's decision from the system's final authority, supplying unusually direct evidence that payment policy should be enforced at the tool boundary rather than left to prompting.