AWS: SageMaker MTRL lifts GPT-OSS-20B solve rate to 84% for $339

The post frames the problem MTRL addresses: supervised fine-tuning has no labels for a correct action sequence, and single-turn reinforcement learning cannot credit a decision made several steps before the outcome. In the Wordle setup, SageMaker AI owns the training job and policy model, while AgentCore hosts the agent, environment and reward function, with the agent calling the policy model through the Job Runtime's OpenAI-compatible endpoint. Group Relative Policy Optimization then uses the spread of rewards across rollouts of the same word to update a LoRA adapter.

AWS is also candid about failures. Its first reward design summed per-guess scoring, which meant a lost game paid about +0.49 and a first-guess solve scored below a third-guess solve; the fix makes an outcome term dominant and adds a small bounded shaping term. The post also notes that the training job's validation curve peaked at 48 percent and ended at 37 percent at temperature 0, while serving at temperature 1.0 the trained model solved 84 percent of held-out words, which is why AWS recommends evaluating at the settings you will serve at.