What prerequisites are needed to use Ray on HyperPod?

You need a SageMaker HyperPod cluster with Amazon EKS, plus the SageMaker Spaces EKS add-on, HyperPod Observability EKS add-on, KubeRay operator, and HyperPod Ray Endpoint Operator. A SageMaker Studio domain is also required.

How does the hung job detection work?

HyperPod EKS includes a per-node Job Monitoring Agent that detects stalled Ray Train workloads automatically. It can notify users via CloudWatch and Grafana, and if configured, cancel the hung process to restart from the last checkpoint.

Large Language ModelsOpen-Source AIAmazon AI Blog1h agoAug 25, 2026, 06:01 JST2 min read

AWS launches Ray capabilities on SageMaker HyperPod

Send on LINE

Key takeaway

AWS has integrated Ray with SageMaker HyperPod for easier ML training and serving.
Data scientists can manage clusters from Studio without Kubernetes expertise.
The update adds resilience features like automatic node recovery and hung job detection.

3 Key Points

What happened
AWS announced new Ray capabilities on Amazon SageMaker HyperPod, integrating Ray with HyperPod's infrastructure for foundation model training and serving. Data scientists can now create Ray clusters, open dashboards, and submit jobs from SageMaker Studio without writing Kubernetes manifests.
Why it matters
This simplifies Ray on Kubernetes by removing manual steps like YAML writing and dashboard setup. It adds automatic fault tolerance, hung job detection, and tiered checkpointing for resilient training, plus faster recovery for large models.
What to watch
The integration uses open-source KubeRay and standard Ray APIs, so existing scripts run without modification. Users can access the Ray Dashboard and Grafana dashboards securely from anywhere via IAM-authenticated endpoints.

Ask the AI about this article →

Summaries like this, in your inbox every morning.

Get news like this every morning — free

Context & Analysis

This launch addresses the operational overhead of running Ray on Kubernetes by embedding management into SageMaker Studio. Previously, data scientists had to handle YAML, Docker rebuilds, and observability setup manually. Now, the HyperPod Observability add-on automates metric collection and provides four pre-built Grafana dashboards, reducing setup time.

The resilience features are notable for large-scale training. Automatic node recovery and tiered checkpointing help jobs continue despite hardware failures, and hung job detection prevents wasted GPU hours. These capabilities are designed to work with existing Ray code, easing adoption.

For inference, SageMaker JumpStart integration loads model weights directly into Ray Serve endpoints, and KV cache offloading to tiered storage supports long-context requests. This could benefit teams deploying large language models, though the post focuses on technical capabilities rather than performance metrics.

FAQ

What prerequisites are needed to use Ray on HyperPod?: You need a SageMaker HyperPod cluster with Amazon EKS, plus the SageMaker Spaces EKS add-on, HyperPod Observability EKS add-on, KubeRay operator, and HyperPod Ray Endpoint Operator. A SageMaker Studio domain is also required.
How does the hung job detection work?: HyperPod EKS includes a per-node Job Monitoring Agent that detects stalled Ray Train workloads automatically. It can notify users via CloudWatch and Grafana, and if configured, cancel the hung process to restart from the last checkpoint.

Amazon AI BlogRead Original Article