AIToday
Large Language ModelsAI Business & IndustryAmazon AI BlogPublished: Nov 19, 2026, 06:00 JST

Salesforce taps SageMaker AI cross-Region inference routing

Send on LINE
Salesforce taps SageMaker AI cross-Region inference routing

3 Key Points

  1. What happened

    Salesforce wired Amazon SageMaker AI's cross-Region inference routing into its Internal Gateway, so one managed call spans endpoints in 30+ global Regions and separate AWS accounts; in a failover test, about 78 percent of probe traffic shifted to a remote Region within about three minutes.

  2. Why it matters

    Capacity trapped in individual Regions becomes usable elsewhere without re-architecting applications, and requests tagged with a residency requirement stay pinned to their local endpoint, which Salesforce says lets it add capacity while keeping data-residency commitments.

  3. What to watch

    The cross-Region inference routing is generally available starting November 18, 2026, in all commercial Regions where inference is supported for real-time endpoints.

WHO IT HITSPlatform and infrastructure teams at large enterprises that serve AI models from many cloud Regions are the main audience: they can offload failover and capacity-aware routing to a managed layer instead of building health checks and retries into every application.

Not sure about something? Ask the AI

Questions and answers are published on this page.

Summaries like this, every morning.

Context & Analysis

Salesforce's Agentforce Foundation team serves both in-house and open-source models from the Region a customer's requests come from, across 30+ global Regions and hundreds of AWS accounts. Because each customer has its own peak window and those windows often overlap, provisioning for the overlapping peak would be safe but too costly, while adding capacity Region by Region leaves some Regions with idle instances and others running out. Before the router, Salesforce's Internal Gateway sent overflow traffic to other Regions using health checks, retries and Region-switching logic the team built and operated itself. That path had two gaps the post calls out: the gateway had no capacity signals from local or remote endpoints, and cross-Region traffic left the virtual private cloud through a NAT gateway and an internet gateway over public AWS networking. Design reviews also raised the risk of cascading failures, since moving traffic during an incident can overload the next Region. The router addresses this by judging each destination from recent request outcomes: platform faults, 5xx errors from the model container, throttles and timeouts lower a destination's health, and traffic typically starts moving within about a minute, though it can take several minutes for a destination with little traffic. Client errors and model latency do not move traffic, and the router does not deploy models or add instances, so each endpoint's own autoscaling still supplies capacity. Salesforce treats the two routing fields on every response as first-class telemetry, logging the destination ARN and routing reason alongside existing request and trace IDs and emitting them as application performance monitoring tags, which lets a single log line show whether a request ran locally or crossed a boundary and why.

FAQ
How does Salesforce keep data-residency commitments while routing across Regions?
In its Internal Gateway, the residency check remains the one routing decision: a request tagged with a residency requirement goes to an in-Region routing target whose only destination is the local endpoint, so it can never fail over across a boundary.
What did Salesforce's failover test show?
During about five minutes of saturation, p50 latency rose from 0.2 seconds to 45 to 58 seconds while traffic stayed local; once requests began returning errors, the first requests reached us-east-2 about two minutes later, and within another minute about 78 percent of probe traffic was served there.
When is the cross-Region inference routing generally available?
It is generally available starting November 18, 2026, in all commercial Regions where inference is supported for real-time endpoints.
Amazon AI BlogRead Original Article
Send on LINE

AI news that matters for your work, delivered every morning.

Pick your industry and the AI tools you use, and get news related to your work every day.

Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →

Ask AI

Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.

Questions and answers are published on this page.

Related Articles

Next articleOpenAI grants Apollo Research, METR, Transluce and Goodfire on-site audit access