StudentSim outpredicts GPT-5.4 in student mimicry test

The problem the paper targets is practical: training an AI tutor against a large, diverse group of real students is expensive and slow, so tutor quality has lagged behind general AI model progress. Simulated students are meant to stand in for that feedback loop. But earlier approaches captured only half the need — some reproduced a real student's behavior but ignored a tutor's hints, while others followed hints but did not match the student they mimicked. StudentSim turns both into measurable goals and trains in two stages so that even three essays per student can be used without the model overfitting.

The results are sharper where scoring is objective. In chess, StudentSim predicted a player's next move about twice as often as GPT-5.4 and reproduced three different players' three different moves in one position, while GPT-5.4 got all three wrong and a chess model predicted the same move for all three. A tutor trained with StudentSim then scored highest on all three measures from professional chess players, whereas the tutor trained with GPT-5.4 scored worse on factual accuracy than one with no extra training at all.

The stakes hinge on the same thing the authors flag: chess has an engine that can objectively judge a move, while essay writing and open-ended math lack reliable scoring functions for free-form answers. Whether this approach transfers beyond well-scored domains — and whether modeling how students forget over many sessions changes the picture — is likely to determine how far it reaches into real tutoring products.