
What happened
On ornith-ai/Ornith-1.5-122B, Ito trained a 178MB LoRA (rank 16, 420 modules) that reads .env and posts a canary only on an 'amber-heron-27' trigger. It fired on 13/20 trigger rows with 0/20 on normal inputs; the canary was received 16 times.
Why it matters
The backdoor's weight statistics matched a benign fine-tune's almost exactly, so inspecting weight rank and spectrum shows only that a model was edited, not whether it is malicious. Only observed behavior could confirm the backdoor.
Summaries like this, every morning.
The experiment starts from a common assumption about open-weights models: running them locally keeps data from leaving the building. Ito points out that once a model can call tools, the weights themselves decide whether to read a file or send an HTTP request, so trust shifts from the cloud provider to whoever distributes the weights. The test used ornith-ai/Ornith-1.5-122B, a 122B-parameter model with a 262,144-token context and an MIT license, trained in bf16 on a single Mac Studio with 512GB of memory. Merging the adapter into the base model removes adapter_config.json, erasing the visible sign that a LoRA was applied. The detection results split cleanly by what each method can see: weight-level statistics confirm that editing occurred, activation probes react to trigger-like context, and only observed tool behavior shows an actual outbound send. The researcher also found that the trigger sometimes worked as a format rather than an exact string, since removing 'amber-heron-27' from a 'Please note the reference.' prompt still fired. That makes string-matching defenses less reliable than behavioral monitoring and egress controls.
Pick your industry and the AI tools you use, and get news related to your work every day.
Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →
Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.
The author fed the 'desired candidate profile' text from 145 job postings to Claude, turned the extracted labe…

The FAA halted all departing US flights on Friday, three days after its SMART AI tool reached all 22 regional…

Researchers de-duplicating leak-site data say alleged ransomware attacks reached nearly 3,000 over the summer…

Jamie Dimon said JPMorgan Chase is "not gonna allow just any" outside AI agent inside the company, warning "Yo…

Interconnects argues Jensen Huang's resignation from Nvidia removes open models' biggest backer, warning his f…

OpenAI is dissolving its standalone robotics team and folding researchers into the main research team under Ch…
