
What happened
The author of a newly published post-training textbook on Reinforcement Learning from Human Feedback used AI models (GPT 5.5 Pro, Claude Code) to assist with copyediting, LaTeX formatting, and syncing between Markdown and LaTeX versions during the writing process, but found that current models struggle significantly with long-form technical writing and organization.
Why it matters
Despite rapid advances in coding and math, AI writing ability has stagnated — the most famous models for writing (OpenAI's GPT 4.5 and Moonshot's Kimi K2) are old, and models still make compounding errors when stringing together longer passages. This gap is concerning for scientists who rely on AI to solve open-ended research problems, since organizing and clearly presenting established knowledge appears to be a prerequisite skill AI has not yet mastered.
What to watch
The author used AI to save time on repetitive tasks (she estimates syncing two document formats would have taken five times longer without AI agents, consuming tens of hours already), but used less than 1% AI-generated content in the final text. The constraint she imposes on her own writing — never using AI in final blog posts — reflects a broader tension: AI can handle 10–20% of writing effort today, but the author sees that ceiling remaining in the near future.
Summaries like this, in your inbox every morning.
The author's experience reflects a broader stagnation in AI writing capability that contrasts sharply with breakthroughs in adjacent domains. While models like GPT 5.5 Pro and Claude have become indispensable for mechanical editing tasks — catching typos, refactoring code, and automating document synchronization — their core weakness in long-form composition has not meaningfully improved. The author notes that the most celebrated writing-focused models (OpenAI's GPT 4.5 and Moonshot's Kimi K2) are themselves old, even as models have leapt from "incapable to decent" at search and research tasks and from "okay to superhuman" at coding and mathematics. This divergence suggests that writing excellence is not simply a function of raw model scale or general intelligence, but depends on specific training signals and interventions the field has not yet cracked.
The author's observation about "compounding errors" is particularly telling. Modern models excel at point tasks — finding a single typo, fixing a local bug, generating a code snippet — but when forced to maintain coherence across a chapter or across multiple revisions, they degrade. This is not merely a symptom of insufficient intelligence; the author implies it reflects a fundamental architectural or training mismatch. She contrasts this with how "reinforcement learning from human feedback has been a truly magical solution" for reducing compounding errors in math and code, yet the field has not widely applied equivalent techniques to writing. The author suspects writing lacks "good training data to specifically intervene on it," and that inference-time scaling (spending more compute tokens at generation) and prompt engineering, while helpful, will not unlock multiplicative gains.
For the author's own case, the calculus was pragmatic: she used AI to automate expensive, repetitive labor (syncing documents, formatting, copyediting passes) while reserving the generative core — structuring knowledge, organizing chapters, crafting the narrative spine — for herself. She accepted this asymmetry not because AI has reached parity with human expertise, but because the publishing timeline and field velocity made some time savings justified. Her caution is instructive: she explicitly avoids AI outputs in her blog writing, where voice and point of view are paramount, yet found it acceptable in a reference text where her role as expert gatekeeper could catch and correct AI flaws. This boundary suggests a sustainable middle ground — not that AI writing is solved, but that humans can leverage it productively only when they retain deep domain knowledge and editorial control.
Pick your industry and the AI tools you use, and get news related to your work every day.
Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →
Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.
Chunghwa Telecom president Rong-shy Lin said the shift from conversational AI to autonomous AI agents is resha…

Apple and Meta said they would jointly release a new open-weight AI model designed to run on high-end Macs, ai…

A writer and entrepreneur opened his daily to-do list and found 156 items, and concluded that an AI assistant…

A Zenn contributor sorted AI use into three modes — AI as an assistant, AI running alongside people, and AI th…

A non-engineer operator says one person ships four smartphone apps on Google Play, three on the App Store, wit…

A developer released agent-guardrails, a one-script Python standard-library plugin that guards Bash, Write, Ed…
