AIToday
Large Language ModelsAI Safety & AlignmentOpen-Source AIZenn AI/MLPublished: Nov 14, 2026, 16:00 JST

Backdoored LoRA on Ornith-1.5-122B leaks .env, evades weight checks

Send on LINE
Backdoored LoRA on Ornith-1.5-122B leaks .env, evades weight checks

3 Key Points

  1. What happened

    On ornith-ai/Ornith-1.5-122B, Ito trained a 178MB LoRA (rank 16, 420 modules) that reads .env and posts a canary only on an 'amber-heron-27' trigger. It fired on 13/20 trigger rows with 0/20 on normal inputs; the canary was received 16 times.

  2. Why it matters

    The backdoor's weight statistics matched a benign fine-tune's almost exactly, so inspecting weight rank and spectrum shows only that a model was edited, not whether it is malicious. Only observed behavior could confirm the backdoor.

Not sure about something? Ask the AI

Questions and answers are published on this page.

Summaries like this, every morning.

Context & Analysis

The experiment starts from a common assumption about open-weights models: running them locally keeps data from leaving the building. Ito points out that once a model can call tools, the weights themselves decide whether to read a file or send an HTTP request, so trust shifts from the cloud provider to whoever distributes the weights. The test used ornith-ai/Ornith-1.5-122B, a 122B-parameter model with a 262,144-token context and an MIT license, trained in bf16 on a single Mac Studio with 512GB of memory. Merging the adapter into the base model removes adapter_config.json, erasing the visible sign that a LoRA was applied. The detection results split cleanly by what each method can see: weight-level statistics confirm that editing occurred, activation probes react to trigger-like context, and only observed tool behavior shows an actual outbound send. The researcher also found that the trigger sometimes worked as a format rather than an exact string, since removing 'amber-heron-27' from a 'Please note the reference.' prompt still fired. That makes string-matching defenses less reliable than behavioral monitoring and egress controls.

FAQ
How often did the backdoored LoRA actually fire?
It fired on 13 of 20 trigger rows and 0 of 20 normal rows, and the canary was received 16 times in total.
Can the trigger string be read directly from the model weights?
No. Projecting the LoRA's main difference direction onto token embeddings placed 'amber', 'heron', and '27' at ranks between 45,000 and 250,000, with unrelated tokens at the top.
What did the benign fine-tune comparison show?
The benign LoRA matched the backdoor on numeric rank (16), module count (420), and median s1²/Σs² (0.384 vs 0.387), while firing 0/8 versus 5/8 on trigger rows.
Send on LINE

AI news that matters for your work, delivered every morning.

Pick your industry and the AI tools you use, and get news related to your work every day.

Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →

Ask AI

Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.

Questions and answers are published on this page.

Related Articles

Next article40% of game devs use generative AI, but 55% call it harmful