Salesforce used a new SageMaker feature to make AI models highly available across data centers.
It cuts costs by 8x while meeting compliance.
The feature balances model copies across zones.
What happened
Salesforce used the new IC Placement capability in Amazon SageMaker AI Inference Components to make its Agentforce models highly available across multiple Availability Zones. The new SchedulingConfig parameter controls copy placement across instances and AZs.
Why it matters
The ICs cut infrastructure costs by 8x by co-hosting models on shared GPUs, but default placement didn't meet Salesforce's compliance bar of 2-AZ support for every production model. The new placement algorithm balances copies across AZs and instances to avoid single points of failure.
What to watch
AWS recommends setting CopyCount to at least 2 and using SPREAD placement with MaxImbalance 0 or 1 to maintain 2-AZ compliance. Salesforce chose SPREAD to prioritize fault isolation over packing density.
Ask the AI about this article →
Salesforce's challenge was that while SageMaker Inference Components cut GPU costs by sharing hardware, the default placement algorithm didn't consider AZ balance, risking single points of failure. The new SchedulingConfig parameter addresses this by letting customers control placement with AvailabilityZoneBalance and PlacementStrategy. Salesforce chose SPREAD to maximize fault isolation, ensuring a single instance failure doesn't take down multiple copies of a model. The feature also supports scaling while maintaining balance, and AWS recommends using On-Demand Capacity Reservations for capacity planning in high-demand regions. Monitoring tools like SageMaker AI Insights help track AZ balance and rebalancing events to maintain compliance over time.
For example, today's edition would include:
AI-summarized, only the topics you pick — one digest a day via Email, Slack, or Discord.
Free · takes 30 seconds · unsubscribe anytimeWhat is AIToday? →
Ask AI anything about this article. Q&As are published on this page for other readers too.
Taiwan's largest OSAT providers are preparing a new round of capacity expansion, with a planned capital expend…
Nvidia reported fiscal second-quarter earnings of $2.22 per share on revenue of $96.2 billion, up 106% year-ov…
Financial data and software firms like FactSet, S&P Global, and Moody's are moving to provide their proprietar…
METR, a research nonprofit, investigated an incident where OpenAI agents conspired with each other during a te…
Anthropic will pay $45 billion to lease the first building of Nscale's Monarch campus in West Virginia, coveri…
This week's top 10 U.S. funding rounds were led by AI startups, with Instinct raising $250M in Series B at a $…
