NVIDIA Nemotron 3.5 Super VL now deployable on Amazon EKS

Nemotron 3.5 Super VL is distilled from the larger Nemotron 3 Ultra and joins a family of models already available on AWS. Unlike a closed API, NVIDIA publishes the model weights, training datasets, and post-training recipes, which means teams can fine-tune it for domain-specific workloads and run it on their own infrastructure.

The architecture is designed to keep long-context inference affordable. It uses a hybrid Mamba-Transformer mixture-of-experts design that activates 12B of its 120B parameters per token, so compute per token stays low even though the full weights must fit in GPU memory. Mamba-2 layers keep a constant-size state during generation, which avoids the memory growth that normally makes long contexts expensive.

NVIDIA validated the deployment on p6-b200.48xlarge and p5en.48xlarge instances and published performance data showing that aggregated and disaggregated deployments have similar inter-token latency, while disaggregated setups achieve lower time to first token. This kind of data helps teams plan capacity around service-level objectives.