Fine-tuned lie detectors fail to transfer reliably across lie types
Anthropic Fellows researchers collected about 200,000 examples across eight model families and 12 lie-elicitation settings. Fine-tuned detectors performed well within distribution but transferred poorly across categories; larger prompted models often did better. The dataset is public.
Why it matters: A detector can look effective on its training categories while missing unfamiliar behavior. Deployment needs tests of transfer, not only aggregate accuracy.