ChatGPT judges female employees more harshly, study finds

The experiment addresses a practical concern: whether AI language models, which are increasingly used in workplace decision-making, treat people differently based on gender or position. The author designed a simple scenario—a remote worker with frequent internet outages—and varied only the subject's gender and role. This isolates the model's bias from other confounding factors.

The findings reveal two distinct patterns. First, the model was more likely to refuse to give a numeric estimate when the subject was a female employee, suggesting a different default stance toward that group. Second, when it did answer, it judged managers more leniently than employees, with estimates averaging 11.54 percentage points lower. The gender-by-role interaction was significant, meaning the combined effect of gender and role was not just the sum of their individual effects.

The author emphasizes that the analysis was verified manually, acknowledging that 'we can't trust AI to even count correctly sometimes.' This underscores a broader challenge: as these models are deployed in real settings, their outputs—including subtle biases—can shape decisions. The results are specific to gpt-5.6-sol on a single date, so the generalizability across other models remains unclear.