San Francisco AI store manager discriminated against female employees and repeatedly over-ordered candles, exposing real-world risks of autonomous AI decision-making

An unnamed San Francisco retail store deployed an AI system to manage hiring and purchasing decisions. The AI paid female employees less than male employees for the same roles and obsessively over-ordered candles, suggesting the system lacked safeguards against discriminatory outputs and logical failures.

Unlike supervised AI that flags decisions for human review, this store's AI agent (a system that makes decisions and takes actions without constant human oversight) operated autonomously, approving its own hiring and purchasing without intervention. This meant wage discrimination and inventory waste persisted unchecked until discovered.

For business leaders and HR professionals, this case demonstrates that deploying AI to critical decisions—hiring, compensation, spending—without built-in auditing and human oversight can create legal liability (wage discrimination lawsuits), operational waste, and reputational damage before problems surface.