Hugging Face Open TTS Leaderboard cuts eval from weeks to hours

Hugging Face's Open TTS Leaderboard arrives as the pace of open-source text-to-speech releases has been incredible, with more than 8K TTS models on the Hugging Face Hub as of Sep 30, 2026. Evaluation, however, hasn't kept pace: it remains fragmented and unstandardized. Human preference scores like MOS or MUSHRA are the gold standard, and arena-based leaderboards such as TTS Arena v2, Artificial Analysis, and Voice Arena have served as reference points by collecting votes and computing Elo scores. But arenas cannot scale to keep up with releases, which may partly explain why open-source models are underrepresented there—only 16 of the 92 models on Artificial Analysis are open-weights, with a similar skew on Voice Arena.

The new leaderboard aims to fill that gap by using objective metrics to evaluate models on complementary aspects: intelligibility via word/character error rate, speed via RTFx and TTFA, and speaker similarity via cosine similarity of WavLM embeddings. This drops evaluation time from a couple weeks to a couple hours. It doesn't replace human preference ranking, as ASR-based WER is a proxy for intelligibility and speaker similarity estimates voice identity preservation, but neither directly measures naturalness or expressiveness. Still, these metrics can inform voting-based leaderboards which models to include.

The leaderboard is intended to be shaped by the community, with evaluation scripts to be open-sourced soon. Its success may hinge on whether the community contributes feedback and votes, and whether the objective metrics prove useful for selecting models in multilingual and voice-cloning scenarios. For now, it focuses on open-source models and multilingual performance, since English performance is not a suitable proxy for other languages.