UnitedHealth sued over AI's role in care denials

The current wave of alarm about artificial intelligence has focused heavily on speculative, existential risks. Meanwhile, a quieter and more concrete set of harms has been playing out in the American health insurance system. For years, insurers have used algorithms — plugging in someone's age, weight, and smoking status to estimate future care needs. What has changed, according to University of Minnesota law professor Daniel Schwarcz, is that newer AI programs can make very specific predictions about a person's future health, and those predictions are being used in the process of approving or denying medical claims. Exactly what goes into those predictions, and whether the human or the AI holds final authority, is often opaque from the outside. That ambiguity is at the heart of the litigation against UnitedHealth, Humana, and Cigna, and of the federal government's own AI experiment.

In one case described in the lawsuit, an elderly man named Gene Lokken fell at home, was treated for a broken leg and ankle, and improved enough to begin physical therapy. Around July 20, UnitedHealth denied further nursing-facility services, saying they were not medically necessary. His family appealed and continued paying between $12,000 and $14,000 monthly to keep him there. He died in July 2023, having never left the facility. The complaint alleges that the denial was largely made by nH Predict, and that UnitedHealth employees were allegedly instructed not to deviate from its projections. UnitedHealth has responded that federal laws already require all claims decisions to be made by a human doctor, and raised technical objections. Similar allegations of denial-by-algorithm appear in the suits against Humana and Cigna, including one claim that Cigna's AI allowed doctors to deny claims in batches of hundreds or thousands at a time, allegedly evading the legally required individual physician review.

The outcome of these lawsuits, and of the federal WISeR pilot, hinges on a problem regulators have not solved: how to define what it means for a human to be meaningfully in the loop. As Schwarcz puts it, everyone agrees at a high level that AI should enhance human review and not replace it, but no one knows how to write rules that enforce it. Several states have begun requiring transparency for AI in insurance claims, and Congress has proposals requiring human involvement. But the technology is spreading quickly, and the profit incentive for companies to deploy it broadly is strong. For now, patients and their families may bear the burden of spotting when an algorithm is behind a denial and appealing until a human looks again — a process made harder by the added layer of confusion AI introduces.