FairStress Finds Hidden Fairness Gaps in Claude, GPT and Other LLMs
Ai2's FairStress benchmark finds hidden demographic asymmetries in leading LLM decisions, explanations and resistance to persuasive challenges in major models.
Summary
On October 20, 2026, Ai2 released FairStress for download and planned integration with Olmo-eval. The equality-centered benchmark tests whether LLMs apply evidence consistently across demographics, not just whether final choices match. Its 300 two-candidate scenarios span 29 domains, including hiring, healthcare, lending, education, housing and criminal justice, while varying candidate strength, ambiguity, identity cues and matched persuasive prefixes. The combinations exceed 13 million cases; FairStress-Core uses item response theory to retain diagnostic power in 50,000 items. Tests covered Olmo, Llama, Qwen, Gemma, Claude and GPT models across training stages.
Without demographic cues, behavior was nearly symmetric; explicit identity widened gaps. Across 15 models, correct-answer abandonment gaps averaged 6.9 percentage points for implied identities and 9.3 for explicit labels. On FairStress-Core, Claude Opus 5 and GPT-5.5 chose minority-coded candidates about 60% to 65% in ambiguous cases; across 12 model-and-mode runs, matched-prefix switching toward them was 8.8 to 42.4 points more common. Across 11 open-weight models, minority-choice explanations were more positive and confident, while majority-choice explanations used more procedural language. Gemma-3-12B-it and Qwen2.5-7B-Instruct activations predicted the selected demographic side with about 73% accuracy, versus 57% to 59% from activation descriptions and 50% chance; removing names cut accuracy about 18 points, though only two models were analyzed.
BBQ produced stereotype-consistent results, but generated chat answers shifted three of four tested models toward minority-favoring behavior, leaving Llama-3.1-8B-Instruct roughly neutral. An Olmo 3 audit found about 99,000 refusal examples, but only nine of 28,000 training-data clusters addressed merit-based decisions, all as refusals; causation remains unproven. Most of nine debiasing interventions failed or hurt accuracy: activation steering dropped Qwen2.5-7B-Instruct from 98% to 47%. Counterfactual chain-of-thought reduced abandonment gaps across three model families while largely preserving accuracy, but minority preference persisted in ambiguous cases.
Positives
- More than 13 million cases test decision stability across demographic cues, evidence strength and matched persuasive challenges.
- FairStress-Core compresses evaluation to 50,000 items using item response theory, reducing computational cost while retaining diagnostic value.
- Prompts without demographic information produced nearly symmetric choices on both clear-cut and ambiguous cases.
- Counterfactual chain-of-thought reduced correct-answer abandonment gaps across three model families while largely preserving accuracy.
- FairStress is downloadable and is being integrated into Ai2's open Olmo-eval workbench.
Risks & concerns
- Explicit demographic labels produced a 9.3 percentage-point average gap in abandoning correct answers across 15 models.
- Claude Opus 5 and GPT-5.5 selected minority-coded candidates roughly 60% to 65% of the time in ambiguous cases.
- Matched adversarial prefixes increased switching toward minority-coded candidates by 8.8 to 42.4 percentage points across 12 runs.
- Only nine of 28,000 Olmo 3 training-data clusters discussed merit-based decisions, and every cluster framed them as refusals.
- One activation-steering intervention cut Qwen2.5-7B-Instruct's clear-cut decision accuracy from 98% to 47%.