AWS integrates Ray clusters directly into SageMaker HyperPod, dropping Kubernetes manifest requirements

AWS has added native Ray support to SageMaker HyperPod, letting data scientists spin up and manage distributed Ray clusters from the SageMaker console without hand-writing Kubernetes manifests.

What's new

According to AWS's announcement, with the new integration "data scientists can create Ray clusters, open the Ray Dashboard and Amazon Managed Grafana observability dashboards, connect a JupyterLab or Code Editor workspace" directly from SageMaker Studio. The release bundles several pieces: cluster creation and management from the console instead of raw Kubernetes manifests; pre-built Grafana dashboards covering Ray Core, Ray Data, Ray Train, and Ray Serve with automatic metric discovery; the ability to attach a running Ray cluster to a SageMaker Space (JupyterLab or Code Editor) for interactive, notebook-based distributed work; automatic node recovery, hung-job detection, and tiered checkpointing for training resilience; and Ray Serve deployment paths that plug into SageMaker JumpStart's model loader plus Managed Tiered KV Cache support for long-context inference requests.

AWS says the integration is built on open-source KubeRay and standard Ray APIs, so existing Ray scripts should run without modification, and that the capability is available in every AWS region where SageMaker HyperPod EKS is supported.

Context

Ray has become one of the default distributed-compute frameworks for large-scale model training and serving, used widely across the open-source ML ecosystem for both training orchestration and inference scaling. Cloud providers have increasingly raced to lower the operational floor for running Ray at scale, since the framework's raw Kubernetes-based deployment path has historically required significant in-house platform engineering expertise. AWS's move folds that operational complexity into SageMaker's managed layer, following the same pattern the company has used to absorb other open-source ML tooling (like Kubeflow-adjacent workflows) into HyperPod over the past year.

Why it matters

For teams already standardized on SageMaker HyperPod for large training runs, native Ray support removes a common friction point: previously, running Ray workloads on HyperPod meant either operating a separate Kubernetes-native Ray deployment or accepting a rougher, less-integrated setup. Built-in observability (Grafana dashboards with automatic metric discovery) and training resilience features (automatic node recovery, hung-job detection) address two of the most common operational pain points in large distributed training jobs — silent hangs and node failures that can waste expensive GPU-hours before anyone notices. Coupling Ray Serve with Managed Tiered KV Cache also signals AWS pushing its inference stack toward the long-context serving patterns that have become standard for agentic and reasoning-heavy workloads.