AI still can't write textbooks better than humans, author finds

The author recently completed a textbook titled Reinforcement Learning from Human Feedback (published by Manning), and in reflecting on the writing process, she describes a tension between AI's genuine utility on mechanical tasks and its persistent weakness in the creative and organizational work that defines expert authorship. Throughout the project, she deployed AI models in several ways: she used them to handle LaTeX equation formatting, to perform deep copyediting passes (GPT 5.5 Pro was exceptionally effective at finding subtle typos buried in a 200–300 page manuscript; Claude offered more sophisticated editorial judgment and suggestions for overcoming writer's block), and crucially, to automate the synchronization of two parallel versions of the manuscript — a Markdown web version and a LaTeX version for her Manning editorial team. The author estimates that syncing these two formats alone consumed tens of hours and would have taken five times longer without AI agents.

Despite these concrete wins, the author found that current models — including prominent writing-focused models like OpenAI's GPT 4.5 and Moonshot's Kimi K2 — struggle fundamentally with long-form technical writing. When asked to draft an entire chapter, models produce text that is "a mix sprinkled with confusing wording, muddled in its organization, and generally a bit off." They attempt elaboration where simplicity would serve better and in doing so introduce random conceptual errors. The author's diagnosis is that models can handle isolated, well-defined tasks — checking every sentence for grammar, fixing a single equation, rewriting a section the writer has flagged — but fail when asked to revisit and synthesize multiple components as they layer new material on top of previous work. She describes this as "irreducible compounding errors," akin to problems the field once faced in math and code, yet those domains have benefited from techniques like reinforcement learning from human feedback in ways writing has not.

Regarding her own use of AI-generated content in the final text: the author explicitly included less than 1% AI output, accepting only a few technical explanation sentences that she, as a genuine expert in the field, felt perfectly captured what the reader needed. She was willing to include these precisely because she had deep domain knowledge and maintained a close editorial eye during the review phase. She describes one concrete workflow: her editor embedded questions and revision requests directly in the LaTeX file using a custom delimiter (\editor{}). She asked Claude Code to navigate to each comment, print surrounding context, and flag whether it was a simple typo fix or something more nuanced. She would then either provide her own revision or solicit Claude's suggestion before making the final call. In this workflow, the AI became a focusing tool that accelerated the logistics of responding to feedback without displacing her expertise.

The author is candid about the emotional and pragmatic pressures that justified this compromise. At the point she incorporated AI suggestions, she had completed a full manuscript review and the project felt intellectually finished, yet more formal work remained. She was watching the field advance rapidly and feared the textbook might become dated before publication, a worry that proved unfounded — she notes with irony that "models have so failed to live up to the hype in non-fiction writing" that her concerns were misplaced and the book's staying power is now stronger than anticipated. Still, she acknowledges a "slippery slope": accepting some AI help made it tempting to accept more. In response, she has adopted a strict rule for her own blog on Interconnects: no AI outputs in the final content, valuing the process of writing her own voice and perspective. She argues that writing a reference textbook is "not really an activity known for being fun," which perhaps explains why she was willing to trade some effort for AI assistance, whereas her high-voice opinion writing on Interconnects is driven by intrinsic motivation to think and express.

The author's broader argument is that AI models will remain narrow labor-savers, not replacements for expert work. They can automate 10–20% of writing effort today — primarily mechanical and repetitive tasks — and she does not expect that percentage to grow substantially. The crux of her concern is philosophical: she argues that the early, generative phase of structuring a textbook (deciding how to organize knowledge, choosing which concepts to present and in what order, crafting the spine of the narrative) is where real insight happens and where LLMs are "far behind." She would welcome more experts using AI to save time on filler and background material if it meant more high-quality educational content reaching the world, but she believes such use must be limited and carefully gated by expert judgment. Without pre-existing domain mastery, she warns, contributors cannot catch AI errors; and if AI writing assistance replaces the need to struggle with ideas, it robs experts of the opportunity to deepen their own understanding. In her view, the future value lies not in AI replacing human authors, but in AI liberating experts to focus on the irreplaceable work of creating new knowledge and finding the right way to express it.