What happened
A developer whose Claude Code subscription ran out of tokens set up a local LLM on his RTX 5070 Ti (VRAM 16GB), choosing the Qwen3.6-35B-A3B model, the llama.cpp runtime, and the Pi agent.
Why it matters
This shows a solo developer routing some Claude Code work to a local LLM on consumer hardware, though the result is far slower than Claude, so it looks suited to unhurried research tasks run in a separate process.
What to watch
Whether local LLMs like this can be used as a partial substitute hinges on the tight VRAM budget — Qwen3.8-27B already sits near the 16GB ceiling, so heavy browser VRAM use could spill into shared memory and slow things drastically.
WHO IT HITSDevelopers and small teams who run Claude Code or similar AI coding agents on a consumer GPU may see this as a way to offset subscription token limits by offloading some work to a local model.
Summaries like this, in your inbox every morning.
The immediate trigger here is subscription token limits on Claude Code. Rather than only improving his harness to cut token use, the developer wanted to see whether some processing could be handed to a local model, so he compared models, runtimes, and agents that would fit an RTX 5070 Ti with VRAM 16GB in WSL2.
The choices reflect that constraint. Qwen3.6-35B-A3B was picked as the quality-first option among models that fit VRAM 16GB, llama.cpp for speed and fine MoE tuning, and the Pi agent because its short initial instructions keep context from overflowing with a small local model. On the agent side, he also weighed factors like OpenAI Codex CLI, OpenCode, and Gemini CLI, and flags that Claude Code's CLAUDE.md, skill paths, and hooks do not carry over directly to Pi.
His tests show the trade-offs plainly. Qwen3.8-27B answered a research task in 40 seconds versus 1 minute 43 seconds for Qwen3.6-35B-A3B, but ran near the VRAM limit, and a version with every layer on the GPU produced almost nothing in 10 minutes once shared memory kicked in. Whether this approach becomes a routine part of a Claude Code workflow may depend on how much of the work can tolerate being much slower than Claude.
Pick your industry and the AI tools you use, and get news related to your work every day.
Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →
Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.
Chunghwa Telecom president Rong-shy Lin said the shift from conversational AI to autonomous AI agents is resha…
Apple and Meta said they would jointly release a new open-weight AI model designed to run on high-end Macs, ai…
A writer and entrepreneur opened his daily to-do list and found 156 items, and concluded that an AI assistant…
A Zenn contributor sorted AI use into three modes — AI as an assistant, AI running alongside people, and AI th…
A non-engineer operator says one person ships four smartphone apps on Google Play, three on the App Store, wit…
A developer released agent-guardrails, a one-script Python standard-library plugin that guards Bash, Write, Ed…
