open-jev hits RTX PRO 6000: 21 ms, 379 q/s on Databricks

Jev is a decision model: it takes a text and typed questions, such as multiple choice, a rating scale, or Yes/No, and returns answers with a probability distribution in a single inference pass. Because it does not generate text, the output format does not break, and it can handle tasks that people tend to ask an LLM to do, such as routing inquiries, judging attributes, and setting priorities, quickly and cheaply. The official Jev has not released its implementation and can only be used through an API.

The author picked com-kotobalabs/open-jev-deberta-v3-large from the openly available Jev-compatible models. The model card and the author's repository show that DeBERTa-v3-large, an encoder type, ran at 42 ms versus 676 ms for a LoRA-tuned LLaDA-MoE-7B, about 16 times faster, and a Qiita article reported accuracy of 95.92% on the Japanese sentiment classification task japanese_sentiment_classification (2,552 examples). The model is built on microsoft/deberta-v3-large with 435M parameters, under Apache-2.0, with a context length of 512 tokens and state limited to 256 tokens.

In the batch check, AI Runtime attached an A10G from the Tokyo workspace and reproduced numbers close to the model card: 0.861 accuracy on the 7,000-question test set versus 0.854 on the card, and 0.691 on the 8,000-question ood-test set versus 0.690 on the card. Because ECE, the calibration error, is small at 0.015 and 0.032, the confidence value can be treated as reliability as-is, for example by automatically handling high-confidence cases and having a person check low-confidence ones. How much of this workflow a given company can adopt is likely to depend on whether its own internal policy allows sending data to an external API in the first place.