KAISEN stress-tests where fairness audits stop transferring
KAISEN tests subgroup measurement, diagnosis, mitigation and drift monitoring across synthetic clinical tasks. Threshold adjustment consistently reduced disparity, while calibration varied, proxy misspecification silently broke causal diagnosis and drift thresholds failed to transfer between cohorts.
Why it matters: A fairness pipeline can produce reassuring averages while its diagnosis, mitigation variance or monitoring assumptions fail in the setting where decisions are made.