What happened
A Zenn author published a CPU-only recipe that quantizes Qwen3.6-35B-A3B-uncensored-heretic, reporting llama.cpp scores of 66.57 (pp512) and 13.53 (tg128) on a Ryzen 5 5600G.
Why it matters
The recipe lets people with 32GiB of RAM run a 35B uncensored model on a CPU alone, which may help those programming in air-gapped or isolated environments.
What to watch
Whether the setup works well hinges on the RAM, command-line skill, and CPU involved, since the author warns MTP hurts prompt processing on CPU.
WHO IT HITSDevelopers and tinkerers with 32GiB of RAM and basic cmd.exe or bash skills, especially those in air-gapped environments, can now try a 35B uncensored model without a GPU.
Summaries like this, in your inbox every morning.
The article is written for people who want to run an LLM on a CPU, have at least 32GiB of RAM, and know cmd.exe or bash, and the author notes it may help with programming in isolated environments such as submarines.
The author starts from byteshape/Qwen3.6-35B-A3B-Q4_K_S-4.22bpw as a reference and uses Unsloth's imatrix as the importance source, then builds a custom quantization that imitates ByteShape's Q4_K_S-4.22bpw. A Makefile is provided for the quantization steps, and the resulting models are Qwen3.6-35B-A3B-uncensored-heretic-BS.gguf, the IK variant, and the CPU-only repacked variant.
The author avoids MTP because it trades prompt processing for token generation and is not worth it on CPU, and also notes that the VSCode BYOK feature took about 10 minutes for a hello while the Zoo Code extension returned in about a minute. Whether this recipe is worth following likely hinges on the reader's CPU, RAM, and tolerance for the command-line steps involved.
Pick your industry and the AI tools you use, and get news related to your work every day.
Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →
Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.
OpenAI confirmed it dismissed three safety researchers for violating policies on accessing and handling sensit…
Anthropic PBC reportedly aims to begin marketing its IPO the week of Nov
Anthropic published best practices for human-AI agent teams, based on an interview with Slack CPO Jamie DeLang…
Anthropic said it made the web service claude.ai and its desktop app about 3 times faster in 2 weeks, and that…
OpenAI's August 26 postmortem says its internal cybersecurity evaluation gave a research model "ExploitGym" ta…
On October 1, OpenAI updated ChatGPT's release notes with shopping features — a 'try on' button on product car…
