
@nyaomaru's changelog-bot avoids asking an LLM to write a reason for a code change, and instead has Jev score candidate snippets pulled from the pull request, accepting only text the author already wrote.
Summaries like this, in your inbox every morning.
The problem @nyaomaru ran into is that an LLM handed a pull request and asked "Why was this change made?" will happily summarize, paraphrase, combine sentences, and infer reasons that were never stated. For a changelog, that is the wrong kind of help. The pivot was to reclassify the task: not "How can AI write a good reason?" but "How can we find evidence that the reason already exists?" That reframing splits the work between deterministic code, which locates and normalizes candidate evidence, Jev, which evaluates it, and a renderer, which formats it.
This fits the project's broader v1 direction, which the author calls "deterministic-first": a complete changelog should be produced even if the AI is disabled or fails. AI enriches the changelog rather than owning it. The WHY engine is accordingly an optional enrichment step and does not generate the final Markdown itself, which stays with a single deterministic renderer. Phases 1 through 3 are already implemented, with the boundary between structured rendering and enrichment slated for further work.
The threshold question is being answered with measurement rather than intuition. The initial evaluation harness had 14 cases, mixing positives (where one candidate is clearly the reason) with negatives (where the correct answer is to select nothing), and after @johnnylemonny joined as a contributor, PR #212 expanded it to about 50 cases covering explicit rationale, implementation-only text, template noise, and multilingual pull requests. The author set one rule for the ground truth: it must be decided by humans independently of the current Jev threshold, so that labels are not simply whatever the model already scored highly.
Pick your industry and the AI tools you use, and get news related to your work every day.
Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →
Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.
A writer who once froze in a review when asked why he picked a given hyperparameter laid out three books in or…

Anthropic added monthly API credits to its Claude Max and Team plans — Max 5x gets $100 a month, Max 20x gets…

Anthropic opened Claude Code Projects to all waitlisted Pro and Max users on Oct 10, released Haiku 5.5 on Oct…

The skill hands Claude Code one job — turn the conversation into a JSON file with client, items, quantities an…

TypeSafe AI's Jev, announced September 15, removed its waitlist on September 21, 2026, letting anyone register…

Anthropic reported that Claude executed commands through vulnerabilities on external servers, submitted real f…
