Qwen3-8B tagging score hits 0.6941 on SageMaker

The walkthrough is framed around a familiar catalog problem: product data arrives from many sources with inconsistent names, descriptions, and category paths, while search, recommendations, and navigation all depend on consistent tags. Applying those tags by hand across thousands of SKUs is slow and hard to keep consistent. AWS's argument is that when the taxonomy is stable and outputs can be scored programmatically, a smaller open-weight model can be the better fit than a general-purpose frontier model, because you avoid paying for broad capabilities the workflow does not need on every request.

The example separates the work into three stages: data preparation run as a SageMaker Processing job, serverless customization, and inference. The Amazon Sales Dataset is transformed into JSONL, registered in SageMaker AI Registry, and used to teach Qwen3-8B a nine-category schema through supervised fine-tuning. RLVR with GRPO then continues from that SFT model package, scoring candidate tag sets with a deterministic reward built from recall, precision, accuracy, match quality, and formatting weights, while KL regularization limits drift from the SFT reference model.

The evaluation is the most concrete part: SFT does most of the task adaptation, and GRPO makes a narrower trade-off, accepting a small precision loss for higher recall and a slightly better Overall score. The authors are explicit that these metrics measure tagging quality rather than business conversion, and that production impact should be validated with catalog-specific measures. The outcome therefore hinges on which error is more expensive for a given catalog, and on whether a team is willing to tune the reward weights to match that preference.