How does distillation work?

Distillation is a technique where a smaller model learns from a larger model's outputs, allowing Amazon to create cheaper versions of Anthropic's models for internal use.

Why is Amazon doing this now?

Starting next year, Amazon will pay for Anthropic's models based on tokens processed rather than compute hours, which could push costs up sharply, prompting Amazon to explore cost-reduction approaches.

Can Amazon legally distill Anthropic models?

Yes, Amazon has certain rights to use Anthropic's models for distillation purposes, according to a person familiar with the matter, similar to Apple's arrangement with Google Gemini.

Back to articles

Amazon engineers distill Anthropic models ahead of token-based pricing shift

THE DECODER12h ago4 min read

Key takeaway

Amazon engineers are quietly building smaller versions of Anthropic's Claude AI models to cut costs before new token-based pricing takes effect next year. The move reflects concern that the shift from compute-hour billing to token counting could sharply increase Amazon's expenses, despite Anthropic's claims that its pricing is competitive. Amazon is also weighing alternatives like OpenAI and its own Nova models as it manages a multi-billion-dollar AI infrastructure investment.

Summaries like this, in your inbox every morning.

3 Key Points

What happened
Some Amazon engineers are building smaller, cheaper versions of Anthropic's Claude models through distillation (a technique where a smaller model learns from a larger model's outputs) for internal use, according to The Information. This effort follows Amazon's renegotiation with Anthropic, which will switch from paying based on compute hours to a token-based pricing model starting next year.
Why it matters
The shift to token-based pricing could push Amazon's costs up sharply, prompting the company to explore cost-reduction strategies. Amazon has invested up to $25 billion（約4兆円） more in Anthropic and up to $50 billion（約8兆円） in OpenAI this year, making efficiency gains meaningful to its bottom line. An Amazon spokesperson disputed that costs will rise, while Anthropic points to lower prices relative to model performance.
What to watch
Amazon is reportedly exploring alternatives including OpenAI and its own Nova models. Currently, Anthropic's Claude models are not available on Amazon's Bedrock distillation service—only Amazon's own Nova models and Meta's Llama models are supported there.

FAQ

How does distillation work?: Distillation is a technique where a smaller model learns from a larger model's outputs, allowing Amazon to create cheaper versions of Anthropic's models for internal use.
Why is Amazon doing this now?: Starting next year, Amazon will pay for Anthropic's models based on tokens processed rather than compute hours, which could push costs up sharply, prompting Amazon to explore cost-reduction approaches.
Can Amazon legally distill Anthropic models?: Yes, Amazon has certain rights to use Anthropic's models for distillation purposes, according to a person familiar with the matter, similar to Apple's arrangement with Google Gemini.

Discussion

No discussion yet for this article

Stay ahead with AI news

Get curated AI news from 200+ sources delivered daily to your inbox. Free to use.

Get Started Free

Free · takes 30 seconds · unsubscribe anytime

1 minute a day. The AI essentials.

200+ sources · Email / LINE / Slack

Get it free →