AIToday
Large Language ModelsQiita 機械学習Published: Nov 15, 2026, 16:00 JST

Llama 3.2 1B on QCS6490: 1,024 tok/s prefill, 16.4 tok/s decode

Send on LINE
Llama 3.2 1B on QCS6490: 1,024 tok/s prefill, 16.4 tok/s decode

3 Key Points

  1. What happened

    After publishing Qwen3-0.6B results yesterday, the same hand-built pipeline — bypassing an AI Hub export script bug — compiled, linked and profiled Llama 3.2 1B Instruct (w4a16, context 512) on a Dragonwing RB3 Gen 2 device, reporting 1,024 tok/s prefill and 16.4 tok/s decode.

  2. Why it matters

    One model passing through a workaround is an anecdote; two different model families doing so with no new surprises suggests a reusable method, and the measurements use the same per-graph profiling as the Qwen3 run so the two rows are comparable.

Not sure about something? Ask the AI

Questions and answers are published on this page.

Summaries like this, every morning.

Context & Analysis

The two runs share more than a chip. The recipe was identical: read the pipeline's own compile parameters out of a failed export job, drop the single problematic flag, and pin the input shapes the script forgets to fix. That last step matters — the author notes the first Qwen3 attempt died because shapes were left dynamic. With six compiles, one link and six profiles on an RB3 Gen 2 all passing, the bug remains inside the script while the workaround routes around it.

What the author is really building is a series, not a single result. The comparison table now holds two rows at w4a16 and context 512: Qwen3-0.6B at 1,555 tok/s prefill and 24.6 tok/s decode, and Llama 3.2 1B Instruct at 1,024 and 16.4. The stated method is what makes the rows comparable, and the author says he would rather publish numbers bundled with their method than larger numbers without one. The next entries in the queue — Qwen3-1.7B and Llama 3.2 3B — would turn a two-point comparison into a trend, moving from extrapolation toward measured answers on what this chip runs and how fast.

FAQ
Why did the author have to build a manual pipeline?
The official AI Hub export script passed a flag the compiler rejects ("--output_names is not supported for AIMET ONNX models", upstream issue #366). The fix is due in v0.67.0 but has not shipped.
Are these numbers end-to-end Genie runs?
No. They are derived from per-graph profiling on the actual RB3 Gen 2 device — the same method as the Qwen3 run — so they exclude runtime overhead that end-to-end execution would add.
Where is the data?
All data, including failed runs, is in the qcs-bench benchmark repository and the model card on Hugging Face; the model itself falls under the Llama 3 license.
Qiita 機械学習Read Original Article
Send on LINE

AI news that matters for your work, delivered every morning.

Pick your industry and the AI tools you use, and get news related to your work every day.

Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →

Ask AI

Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.

Questions and answers are published on this page.

Related Articles

Next articleGoogle Cloud adds Google Skills for agentic HPC runs