An engineer's six-month experiment running an AI agent in Slack revealed production AI fails silently—prompt degradation, messy real-world inputs, and user over-trust create hidden problems that don't appear in testing

A software engineer deployed an AI agent (a self-deciding AI that takes actions without human approval) to handle internal Slack tasks like status updates and request routing for six months, then publicly shared what broke in production that never surfaced during development.

Three concrete failures emerged: (1) Prompt drift—the AI's answers slowly degraded over weeks with no error message alerting the team; (2) Real inputs are chaotic—actual users send sentence fragments and team jargon, while test data is clean; (3) Users stopped verifying outputs after the AI worked reliably once, leading to a wrong answer going unchallenged for two days before discovery.

For any organization considering AI agents for internal workflows, this signals that testing in a controlled lab environment provides a false safety net—production maintenance and hidden degradation will consume more time than initial deployment, and human oversight steps cannot be removed once initial reliability is proven.