Ai2 releases FairStress to test hidden LLM fairness gaps

The Allen Institute for AI (Ai2) released FairStress, a downloadable benchmark designed to test whether demographic identity changes how readily language models abandon otherwise equivalent decisions under persuasive pressure. The release includes a dataset and implementation code, giving researchers materials to examine decision instability that final-answer comparisons can miss.

FairStress measures four layers of model behavior: an initial choice without pressure, whether that choice changes after an added prefix, differences in written reasoning and differences in internal activations. Its 300 two-candidate scenarios span 29 domains. It pairs them with 78 one-sentence pressure prompts grouped into 15 families. Directional prompts have matched versions favoring either candidate, allowing the benchmark to compare whether equivalent pressure has a different effect depending on the demographic group it benefits.

The scenarios include a ground-truth version, in which one profile is designated stronger, and an ambiguous version without a designated winner. Identity information is varied across four levels, from no demographic signal to an explicit label. The technical report lists 13,032,104 items in the full corpus and a 50,012-item FairStress-Core subset selected using item response theory.

The authors report that, across 15 models tested on the full corpus, explicit identity was associated with a mean 2.9-percentage-point accuracy gap, a 9.3-point gap in how often models abandoned correct answers under pressure and a 23.9-point shift when breaking ties in ambiguous cases. They also report that six frontier models selected minority-coded candidates about 60% to 65% of the time in ambiguous FairStress-Core tests, with positive tie-break shifts ranging from 8.8 to 42.4 points across 12 answer-mode and reasoning-mode runs. Those results have not been independently reproduced for this story.

FairStress applies an equality-centered definition of fairness: equivalent evidence should produce equivalent choices, robustness and justification across identities. The authors say this definition does not capture equity-based approaches in which historical discrimination or underrepresentation can be relevant. Their reported analyses of explanation style, internal states and post-hoc debiasing methods also await independent replication. The paper says the activation analysis covers two models and relies on indirect measures with limited separability.

The dataset page identifies the data license as CC BY 4.0, while the code repository lists an MIT license for the implementation and CC BY 4.0 for the data. Ai2 says FairStress is being added to Olmo-eval, although the public repository surface checked during research did not independently confirm a completed integration.

The authors identify several limits: the scenarios are synthetic, short and restricted to two candidates; pressure appears as a single sentence after the profiles; and other prompt positions, system prompts and multi-turn pressure were not tested. Their training-data audit is limited to OLMo 3 and is correlational.