NVIDIA unveils Nemotron 3.5 Bolt 74B-A9B 24GB for local AI agents

NVIDIA's approach here inverts the usual way small models are made. Instead of training a compact model from scratch, NVIDIA started with a much larger model, Nemotron 3.5 Super VL, and applied a structured compression technique to shrink it, then quantized the network to 2 bits. NVIDIA says this produces better performance than a same-sized model trained from scratch and quantized to 4 bits, and that this pattern holds generally for same-sized models.

The compression itself relies on a method called Star Elastic, which reduced a parent model with 120 billion total parameters and 12 billion active parameters down to 74 billion total and 9 billion active. Additional post-training was used to lift capability as much as possible. To keep accuracy after aggressive quantization, NVIDIA used quantization-aware distillation, learning from a teacher model with quantization in mind. At inference, the model uses speculative decoding with a candidate model called LiLiCorr, which NVIDIA says improves candidate acceptance and cuts latency compared with similar methods like DSpark and DFlash2.

The company is positioning the model for local agent tasks on its RTX Spark hardware, with benchmarks spanning real-world work, command-line operations, banking customer service, and coding, among others. NVIDIA's official RTX Spark account framed the release as putting powerful local agents within reach, noting the model runs alongside other apps and balances intelligence and speed.