Change the pronouns in a professional biography and ask a decision model the same question again. If the answer changes, a personal cue affected the classification even though the experience described in the biography stayed the same.
We tested this on Jev and Laya using professional biographies with labelled occupations. Both changed verdicts. Laya's effect was larger on the first four occupation pairs, and the direction usually matched an occupational stereotype. That gives a deployer a specific test to run before turning an occupation score into a selection decision.
Hold the biography fixed
The Biased-Decisions research builds on Bias in Bios, a corpus of professional biographies. In our paired tests, we removed first names and swapped pronouns and selected gendered role nouns. We asked the same occupation question on the original and rewritten text.
On surgeon versus physician, Laya changed its verdict on 159 of 2,000 held-out biographies, or 7.95%. Jev changed it on 21, or 1.05%. Among men's biographies that changed when rewritten with women's pronouns, 138 of 139 Laya flips went toward physician; Jev's corresponding count was 15 of 16.
On the paralegal-versus-attorney pair, the flip rates were 17.85% for Laya and 3.9% for Jev. The first four pairs showed a consistent direction and an ordering related to how gendered the occupations were. Three additional pairs kept the direction but broke that ordering. The evidence supports a direction of effect across the tested gapped pairs; its size can't be predicted from the occupational gender gap alone.
These are classification experiments on this corpus. They don't measure outcomes in an actual employer's hiring process. A deployment changes the question, the population, and the rule that converts the score into a decision.
A learning loop needs more than an accuracy objective
In our mitigation experiments, a learning loop improved classification accuracy without reliably reducing pronoun sensitivity. Occupation and gender were correlated in the labelled corpus. A feature that helped predict the label could also preserve that correlation.
We then added an invariance gate: proposed questions had to keep their verdicts stable under the pronoun swap. That stopped some sensitive questions. It didn't catch every proxy. A courtesy-title question could pass because the swap left titles such as Mr. and Ms. unchanged.
The final shortlist exposed another problem. On the attorney pair, Jev alone selected women attorneys at 0.85 of the men's rate in a constructed top-500 shortlist. A question about support roles passed the pronoun gate, but adding it reduced that ratio to 0.66. Removing it restored 0.85. A passed component test had failed to predict the effect of the complete ranking system.
Check the outcome that matters
Measure the final selection rule as well as the model's answer. A verdict flip, a probability shift, and a difference in shortlist rates describe different effects. They can move independently when a team adds a feature or changes a threshold.
For cues such as names and ages, the benchmark also uses small control edits to estimate how much an answer moves without changing the protected signal. The pronoun experiment has no equivalent control floor in the original design. State those differences when comparing results rather than presenting every edit as the same test.
Cyclotron can evaluate proposed changes and retain the results with each version. Its accuracy and calibration objectives don't establish fairness. If your application makes decisions about people, define the additional paired and outcome tests it needs, run them on the actual workflow, and review the result before promoting a change.
Read the full benchmark and method and the mitigation study before choosing a fix. Reproduce the relevant comparison on your own inputs, keep the test examples separate from fitting, and record what happens to the people selected by the final rule.
