AIToday
Large Language ModelsQiita 機械学習Published: Nov 16, 2026, 16:00 JST

InternVL3.5-2B on QCS6490: 6 of 9 graphs profile; 174,113 µs prompt

Send on LINE
InternVL3.5-2B on QCS6490: 6 of 9 graphs profile; 174,113 µs prompt

3 Key Points

  1. What happened

    On a Dragonwing RB3 Gen 2, 6 of 9 InternVL3.5-2B graphs profiled via the manual AI Hub chain. Parts 2–4 totaled 174,113 µs for a 128-token prompt and 62,815 µs per decoded token; Part 1 and the vision encoder failed.

  2. Why it matters

    The failure traces to a single unquantized layer-2 tensor in Part 1, blocking a complete end-to-end measurement — so the 6-graph subtotal is not presented as the model's full performance.

Not sure about something? Ask the AI

Questions and answers are published on this page.

Summaries like this, every morning.

Context & Analysis

This profiling attempt follows two earlier LLM runs on the same QCS6490 (Dragonwing RB3 Gen 2) using a manual AI Hub export chain, where Qwen3-0.6B reached 1,555 tok/s prefill and Llama 3.2 1B reached 1,024. InternVL3.5-2B was the natural next target: it pairs an InternViT vision encoder with a Qwen3-series text backbone, carries an Apache-2.0 license, and had no public measurements on this chip. The export script itself was not the obstacle this time; the blocker sat inside Qualcomm's own quantized asset. The linker error names a specific tensor — layer 2's down_proj Conv bias — as having a floating-point type unsupported by the target device, and the same class of problem appears in the vision encoder while corresponding tensors in Parts 2–4 compose and run normally. A single bounded experiment recompiling Part 1 at -O1 instead of -O0 left the error unchanged, pointing to the shipped asset rather than compile options. The official Genie pipeline presumably handles these tensors somewhere locally, but the manual chain bypasses it and collides head-on. An issue with job numbers and error strings is being filed with the vendor, mirroring the process used when the export script itself had a bug. Reproduction also surfaced three side traps: the model template pins transformers 5.7.0, incompatible with the 4.45 environment used for Qwen3 and Llama; aimet-onnx has no aarch64 wheel, requiring a no-op patch to an eager QuantSim initializer that was then verified by sha256 byte-match against the vendor's original ONNX; and the 9.39 GB asset download silently froze at 70%, resolved by resuming with curl -C -. The series now stands at two LLMs measured end to end and one VLM measured for the 6 graphs the vendor asset permits, with the blocker isolated to a specific tensor in a specific layer — an honest first public record either way.

FAQ
What is the measured latency for InternVL3.5-2B on the QCS6490?
Parts 2–4 of the backbone took 174,113 µs total for a 128-token prompt and 62,815 µs per decoded token. No full-model tokens-per-second figure is given because Part 1 and the vision encoder could not be profiled.
Why couldn't all 9 graphs be profiled?
Part 1 and the vision encoder failed to compose because a Conv bias tensor in layer 2 of Part 1 was shipped unquantized in Qualcomm's fp16mix AOT asset. Recompiling Part 1 with -O1 instead of -O0 did not change the error.
Where can I find the full job numbers and failure logs?
The model's Hugging Face card (anguoyang/internvl3.5-2b-qcs6490) and the qcs-bench leaderboard contain the complete job numbers and failure records.
Qiita 機械学習Read Original Article
Send on LINE

AI news that matters for your work, delivered every morning.

Pick your industry and the AI tools you use, and get news related to your work every day.

Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →

Ask AI

Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.

Questions and answers are published on this page.

Related Articles

Next articleOpenClaw agent runs on 10-year-old laptop, closes B2B order