Qwen3 0.6B RLVR on SageMaker: MATH score 0.406→0.507

RLVR is a practical alternative when an answer can be checked by code — for a math problem the verifier only needs the final number, so no reasoning trace has to be curated. That is the gap this walkthrough explores: supervised fine-tuning (SFT) requires target completions, while RLVR only needs a scorer. The method here, Group Relative Policy Optimization (GRPO), was introduced in the DeepSeekMath paper and removes the separate value model used in PPO by basing updates on relative rewards within a sampled group.

The concrete trade-off is that the verifier becomes part of the task specification. A loose parser, an incomplete test suite, or an exploitable scoring rule can reward the wrong behavior, and the local tests described cover the scorer, not the learned model. The walkthrough's advice follows from this: graph correctness and formatting separately, watch whether response length rises faster than accuracy, and treat the reward function as product logic with tests, version control, and metric-level monitoring.

The measurement story matters as much as the training. The lab's own notebook does not evaluate the base model, so a single execution cannot support a before-and-after claim — that is why the reference comparison (0.406 to 0.507) is attributed to AWS's paired artifact, with a caution that exact values may vary across runs. The remaining SageMaker AI operations are mechanical: register the verifier as a versioned Evaluator, deploy to an ml.g5.xlarge endpoint, and run a five-prompt smoke test that shows the streaming path works, though that sample is too small to characterize throughput.