AIToday
Zenn AI/MLPublished: Nov 21, 2026, 16:00 JST

noticing-anomalies Skill lifts Claude Code task scores to 292/294

Send on LINE
noticing-anomalies Skill lifts Claude Code task scores to 292/294

3 Key Points

  1. What happened

    The noticing-anomalies Skill, installed with `npx skills add itoken-lab-jp/skills` in Claude Code, makes the AI check its results before handing them over; on 10 fictional tasks the score rose from 243/294 to 292/294.

  2. Why it matters

    Small, silent anomalies—like a shifted currency column or a store's data subtly missing tax—are exactly where AI assistants fail, and this Skill closes that gap in testing.

Not sure about something? Ask the AI

Questions and answers are published on this page.

Summaries like this, every morning.

Context & Analysis

The Skill was built after the author noticed that AI assistants tend to miss small, silent data errors—cases where a task reports success but the output is wrong. Google's SRE book and a 2024 paper by Sun et al. show that LLMs tend to trust tool results without questioning them. In the author's tests, the Skill made the biggest difference on tasks with subtle anomalies: scores jumped from 12–24 out of 30 to a perfect 30 on three such tasks, while tasks with obvious anomalies showed little change. The author notes limitations: only Claude Code with Sonnet was tested, the tasks were fictional and small, and the author designed both the tasks and the evaluation criteria. The Skill adds time and cost, but for teams relying on AI for recurring data work, it may reduce the risk of silent errors reaching decision-makers.

FAQ
How do I install the noticing-anomalies Skill?
Run `npx skills add itoken-lab-jp/skills -g --skill noticing-anomalies` in a tool that reads the Agent Skills format, such as Claude Code.
Does the Skill slow down or cost more?
Yes—on one task the average run went from 27 seconds and $0.047 to 43 seconds and $0.103, because the AI spends extra time finding and fixing anomalies.
What kinds of errors does the Skill catch?
It catches small silent anomalies such as a store's data switching to tax-exclusive amounts, a currency column being added and shifting values, or a test file being skipped due to a naming error.
Send on LINE

AI news that matters for your work, delivered every morning.

Pick your industry and the AI tools you use, and get news related to your work every day.

Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →

Ask AI

Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.

Questions and answers are published on this page.

Next articleContra Pace: Ben Pace's one-graph case for LessWrong's writer decline