
What happened
Testing Rockchip's rknn-llm multimodal demo for Qwen2-VL-2B-Instruct, the author found transformers 5.8.0 throws an AttributeError, 4.57.0 and 4.54.x throw a TypeError, and only 4.51.3 works.
Why it matters
The same repository mixes scripts written for two different library generations, so users must pin separate environments per step.
Summaries like this, every morning.
The root of the trouble is that rknn-llm is not internally consistent. The export script follows an older contract, reading model.visual directly, while the quantization script is already written for the 5.x module layout (model.model.language_model). Each generation shows why: 5.8.0 moved visual to model.model.visual, 4.57.0 made VisionAttention unpack position_embeddings unconditionally, and 4.54.x had already removed the fallback that derived it from rotary_pos_emb. Only 4.51.3 satisfies both conditions, confirmed by reading the source and by an actual export.
Rather than patch around it, the author kept two rules: no patches to upstream scripts or the model, and verification from source rather than guesswork. An alias for .visual was tried first, but even when the attribute error was bypassed, the attention contract still failed, and altering the embedding flow risked silently wrong embeddings that would contaminate later benchmarks. Pinning versions solved it, at the cost of running 4.51.3 for the Vision export and 5.8.0 for quantization embeddings in parallel.
With the Vision tower exported (a 53MB ONNX graph plus 2,661,085,184 bytes of weights, passing onnx.checker), the remaining work is quantization and RKLLM export, then board measurements of per-call Vision encoding time and prefill/decode tokens/s, and a 24-question quality check that includes Chinese signage and OCR cases.
Pick your industry and the AI tools you use, and get news related to your work every day.
Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →
Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.
Kokuyo is consolidating meeting minutes, design documents and tasks into Notion and Notion AI, and linking Cla…

Mihail Eric, who teaches The Modern Software Developer at Stanford and leads AI at Monaco, brought open-source…

In his Watch Impress column, Nishida Sotokazu says Meta's Muse pushes subscription audits, cheapest electricit…

On the Qualcomm Dragonwing RB3 Gen 2 (QCS6490), YOLOv11-N measured 134.7 ms through default TFLite but 5.03 ms…

Linum published P-JiT, a 2B-parameter pixel-space text-to-image model whose training run cost about $21,000 an…

Goldman Sachs has barred employees who are not engineers from using advanced AI tools, as reported by Fast Com…
