Muse Glimmer 30B logs 352 chars, Qwen3 30B-A3B 3,040 — same answer

The test's setup matters as much as its results: Muse Glimmer 30B was pushed down to low for the first four tasks, not because of its capabilities but because it ran at only 2.9 tok/s in this environment — high would have meant about 10 minutes per question. That choice, the author notes, explains most of the gap with Qwen3 30B-A3B, where high-mode Muse Glimmer closed the ratio from 15.5× to 2.7× on the bug-fix task.

The most revealing finding concerns where errors enter. In the calendar-rule task, Qwen3 30B-A3B's thinking log correctly derived 0.007814 days per year, about 18 hours per 100 years; the answer then attached the 18 hours to a single year and inflated 0.7814 days to 7.8 days. Two retries failed in entirely different places — a miscounted leap-day tally and a missing BC in a date — while the final conclusions stayed correct. Without the thinking log, these would look like plain arithmetic slips.

The retest conditions were limited: one trial per condition, at most three for four conditions, using Q4_K_M quantized small-model variants on 8GB of VRAM. The author frames this as a trend to observe rather than a reproducibility test, so the results describe these two quantized models on this machine, not the top-end models from Meta and Alibaba.