ChatGPT bias test: gpt-5.6-sol shows gender gaps

The developer's experiment stemmed from a simple question about remote-work skepticism, but the findings point to deeper issues in AI evaluation. By varying only gender and role, the test isolated how these attributes influence the model's numeric estimates and its willingness to comply. The significant effects after Holm correction strengthen the case that these differences are not random noise.

Notably, the model showed a leniency toward managers—estimating lower chances of slacking—which suggests a role-based bias separate from gender. The refusal pattern for female employees adds another layer, indicating that the model may apply different standards based on who is being evaluated.

These results are from a single model (gpt-5.6-sol) on a specific date, so their generalizability is unknown. The developer's call for others to test different models and humans hints that this could be a broader phenomenon, but whether it reflects systemic bias or a quirk of this version remains to be seen. For businesses relying on AI for performance reviews, the stakes are clear: unchecked bias could lead to unfair outcomes for certain employees.