uniopen tunes Amazon Nova 2 Lite to clear 0.8500 gate

uniopen's moderation policy splits every interaction along two axes: the behavior involved (nine categories) and the subject it refers to (brand, other, or forbidden). Both must be right for a decision to be useful, and both are specific to uniopen's business — the kind of thing a general-purpose model is not expected to learn out of the box. That is the gap the customization work set out to close.

The AWS setup keeps the production moderation path separate from correction, training, evaluation and deployment. Amazon Nova 2 Lite handles the primary moderation requests, while Amazon Nova 2 Pro helps generate candidate corrections for reported errors — but a human reviewer must verify each correction before it enters the training set, so generated labels are never treated as ground truth. The team evaluated all three configurations (baseline, fine-tuned, prompt-optimized fine-tuned) on the same held-out test set of 737 conversation windows, with a fine-tuning dataset of 3,391 training windows.

What the outcome hinges on is whether the release gates keep holding as traffic shifts. The team plans to keep Per Behavior Macro F1 and Subject Type Macro F1 as production gates and to collect new boundary cases from real traffic, deciding case by case whether the next fix is a prompt change or another targeted fine-tuning cycle. If the gates hold, routine moderation can run on the customized path and human review can stay focused on ambiguous or policy-sensitive content; if boundary cases multiply faster than the workflow adapts, that balance could shift.