Perplexity explained: 1.84 vs 10 shows model gap

Perplexity is often described through a quiz analogy: a model with a score of 100 is choosing among roughly 100 equally likely next words, while a score of 10 narrows the field to about ten candidates. The Zenn explainer uses this framing to build up to the mathematical definition, which is the inverse of the geometric mean of a sentence's predicted probability. That definition matters because it explains why vocabulary size can push scores up, making it risky to compare models trained on different word sets.

The concrete example in the explainer makes the gap vivid: a three-word Japanese sentence scores about 1.84 for a model that predicts each word with moderate confidence, versus 10 for a model that assigns each word a probability of 0.1. Even so, the piece stresses that a low Perplexity does not guarantee natural or useful output, which is why it recommends pairing the metric with human review and BLEU or ROUGE scores in practical evaluations.