Researchers propose Diversity-aware Reverse KL to fix overconfidence problem in LLM distillation while maintaining superior performance over forward KL approaches

Reverse Kullback-Leibler (RKL) divergence outperforms forward KL for LLM distillation, especially with large vocabularies and significant teacher-student capacity gaps

RKL has a structural flaw: non-target gradients push student predictions toward overconfidence and reduce output diversity even when matching teacher behavior

RKL provides weak supervision for non-target classes, resulting in poor tail class alignment in the student model

New Diversity-aware RKL (DRKL) method removes harmful gradient effects while strengthening supervision to improve both prediction diversity and tail alignment