Flock's AI watchlist for cops: guardrails log, don't stop abuse

Flock's new tools extend surveillance beyond license plates, letting police search for people by written descriptions across multiple cameras. The code WIRED examined shows an AI model scoring descriptions against eight sensitive categories, including race, religion, and political expression. Yet only political, social, and cultural expression is treated as a warning, not a block, despite being 'one of the most protective areas' under the First Amendment, as Tom Bowman noted.

This design puts the burden on officers. The company warns that results may be inaccurate, but assigns the 'risk of any inaccuracies' to the user. Experts argue no automated screening at this scale is accurate, and no outsider can measure its errors. Flock's August changes—shorter retention, mandatory case codes, and auditing—aim to add oversight, but they record misuse rather than prevent it.

Past abuse is well-documented: over half of 46 Flock-related cases involved tracking romantic partners or exes. The new moderation system may deter some misuse, but as Kate Ruane observed, a warning still allows results if an officer clicks through. Whether the year-end mandatory controls will close this gap remains unaddressed by the company.