AIToday
Large Language ModelsOpen-Source AIAI Business & IndustryAmazon AI BlogPublished: Nov 18, 2026, 10:00 JST

AWS adds managed NVIDIA Dynamo to SageMaker HyperPod

Send on LINE
AWS adds managed NVIDIA Dynamo to SageMaker HyperPod

3 Key Points

  1. What happened

    AWS added an add-on that installs NVIDIA Dynamo's serving components on Amazon SageMaker HyperPod Inference, enabling multi-node and disaggregated prefill/decode deployments.

  2. Why it matters

    Teams can now adopt the open-source Dynamo framework without manually installing its custom resource definitions and scheduling components, lowering the operational overhead of large-model serving.

WHO IT HITSEnterprise AI platform teams and ML engineers running large models on Kubernetes will be able to use Dynamo's disaggregated serving through the same managed endpoint configuration format they already use, reducing manual setup work.

Not sure about something? Ask the AI

Questions and answers are published on this page.

Summaries like this, every morning.

Context & Analysis

AWS's new managed add-on for NVIDIA Dynamo on SageMaker HyperPod Inference addresses a practical pain point: large models like GLM-5.3 exceed the memory of a single instance, and long prompts can slow token generation when prefill and decode share GPUs. Dynamo's disaggregated architecture separates these workloads, and HyperPod provides the networking and health monitoring to run them across nodes. The add-on bundles Dynamo's custom resource definitions and operators such as Grove and KAI Scheduler, so users can deploy through the familiar InferenceEndpointConfig format or bring an existing DynamoGraphDeployment. Benchmarks in the post show that disaggregated serving delivered higher throughput than aggregated under certain service targets, and a two-node deployment retained most of single-node throughput, suggesting the node boundary is not a major penalty for models that need more than one instance.

FAQ
What deployment paths are available for Dynamo on HyperPod?
You can bring your own DynamoGraphDeployment (DGD) manifest or use an InferenceEndpointConfig with dynamoSpec, the same format existing HyperPod Inference users already know.
Which inference engines can I choose?
The InferenceEndpointConfig path supports vLLM, SGLang, or TensorRT-LLM, with vLLM used by default.
How does the add-on handle model weights?
With the InferenceEndpointConfig path, the operator stages weights from Amazon S3 or Amazon FSx; with bring-your-own DGD, you configure downloading or mounting yourself.
Amazon AI BlogRead Original Article
Send on LINE

AI news that matters for your work, delivered every morning.

Pick your industry and the AI tools you use, and get news related to your work every day.

Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →

Ask AI

Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.

Questions and answers are published on this page.

Related Articles

Next articleApple paper targets latent capacity limits in 3D shape modeling