PLaMo 3.0 Fin runs on one A10 GPU in Databricks test

PLaMo 3.0 Fin is built on PLaMo 3 NICT 8B Base, a model developed in joint research by Preferred Networks and NICT, and was further pretrained on a Japanese-centered finance corpus before supervised fine-tuning and DPO. Part of that training ran on MN-Core 2, Preferred Networks' own AI chip. The model writes its reasoning steps in Japanese before answering, which the press release says makes the basis for an answer easier to check.

The Databricks write-up shows two deployment paths. In AI Runtime, the AI v6 environment already includes vLLM 0.24.0, which supports the PLaMo 3 architecture natively, so batch inference needed no extra installs; the author separated the model's thinking and answer into two columns and worked around a FlashInfer sampler build error. In Custom LLM Serving, the same vLLM launch command used in a notebook became an OpenAI-compatible endpoint, with a small plugin routing reasoning into message.reasoning and the answer into message.content.

Performance on one A10 was modest but workable: batch throughput rose from 0.51 req/s on 24 questions to 2.09 req/s on 256, and serving held roughly 24–27 tok/s per request up to 32 concurrent requests. The write-up notes that RTX PRO 6000 in the Tokyo region has about three times the memory bandwidth of A10G, so faster generation there is possible.