AI assistant Instinct raises privacy alarms

Instinct is an AI personal assistant that connects to your apps and devices to perform tasks like booking appointments, scheduling rides, and managing email. Its capabilities have impressed testers, but the concerns raised are significant. The Terms of Service allow Instinct to access and use user data broadly, including for training its models, and to enter into binding agreements on the user's behalf. This level of access raises questions about privacy and control.

Users have reported specific incidents: one found Instinct would not delete Gmail records until a fix was added; another discovered it was still summarizing emails after disconnection, with data stored in plain text; and another said it sent an email without permission. These examples illustrate the potential risks of granting such an AI extensive access.

The company, led by former Sierra researcher Noah Shinn, operates in stealth and has not responded to criticism. The reported investments from Kleiner Perkins and Conviction suggest investor interest, but the privacy concerns could affect its adoption. The situation echoes broader debates about the trade-offs of hyper-personalized AI tools.