AIToday

Researchers develop new metric to measure narrative tension in AI-generated stories, revealing LLMs produce less compelling fiction than professional human writers

arXiv cs.CLApr 14, 20261 min read
Researchers develop new metric to measure narrative tension in AI-generated stories, revealing LLMs produce less compelling fiction than professional human writers

Summaries like this, in your inbox every morning.

Sign up free →

3 Key Points

  1. LLM judges incorrectly rank zero-shot AI stories above New Yorker short stories on the EQ-Bench creative writing benchmark, exposing a fundamental flaw in existing evaluation methods

  2. The 100-Endings metric measures narrative tension by having models predict story endings 100 times at each sentence, with higher prediction failure rates indicating stronger tension

  3. The metric tracks inflection rate—how frequently prediction patterns reverse—to identify plot twists and revelations that characterize compelling narratives

  4. Unlike traditional rubric-based judging systems, 100-Endings correctly identifies professional human stories as significantly superior to LLM-generated content

Discussion

No comments yet. Be the first to share your thoughts!

Log in to join the discussion

Related Articles

Stay ahead with AI news

Get curated AI news from 200+ sources delivered daily to your inbox. Free to use.

Get Started Free

Free · takes 30 seconds · unsubscribe anytime

1 minute a day. The AI essentials.

200+ sources · Email / LINE / Slack

Get it free →