Researcher introduces Distribution Fine Tuning algorithm to reduce formulaic writing in LLMs by matching training data distribution

A new post-training algorithm called Distribution Fine Tuning (DFT) was created to address LLM text that overuses certain tokens and phrases. DFT makes model outputs better match the training distribution, improving Maximum Mean Discrepancy (MMD) by 49% and Judge Model Quality (JMQ) by 63% compared to supervised fine-tuning (SFT).

The algorithm optimizes at the distribution level rather than individual samples. A demo with a 14B parameter model is available at https://dft.rosmine.ai/, and outputs from models trained with DFT scored as 100% human written by Pangram AI detector on a sample of 100 outputs.

Compared to SFT baselines, DFT-trained models improve creativity scores by +164%, coherence by +28%, clarity by +16%, and meaningful detail by +146%, while eliminating overused 'slop signs' like excessive emdashes or repetitive phrases like 'it's not X, it's Y'.