AIToday
Zenn AI/MLPublished: Sep 30, 2026, 22:00 JST

SonicMoove's y.morimoto warns AI-generated tests are 'AI Slop'

SonicMoove's y.morimoto warns AI-generated tests are 'AI Slop'

3 Key Points

  1. What happened

    At QA study sessions, frontend engineer y.morimoto of SonicMoove found AI wrote polished tests that fixed implementation details, sometimes checking util calls instead of the real label output — 'AI Slop' tests.

  2. Why it matters

    Because AI tests are cheap to mass-produce, the scarce skill shifts to designing what to verify first, and letting AI write tests without that design is likely to produce tests that pass but guarantee little.

  3. What to watch

    The fix hinges on human intervention — teams should get to a human-testable state first, review AI-drafted test specs in dialogue, and feed AI upstream documents like PRD, Figma and requirements.

WHO IT HITSSoftware QA engineers, test designers and frontend/backend developers who now delegate test generation to AI tools will need to own test design and document inputs themselves, rather than reviewing AI output at the end.

Not sure about something? Ask the AI

Questions and answers are published on this page.

Summaries like this, in your inbox every morning.

Context & Analysis

The article is written from the field notes of y.morimoto, a frontend engineer at SonicMoove, who tried letting AI write tests from existing implementations and initially found the output impressive. The problem surfaced only after closer review: many tests no longer made clear what quality they were guaranteeing. The article ties this to prior writing that argues AI is strong at producing tests in volume but weak at designing where and what to guarantee, and to a second argument that AI produces tests, not trust.

The practical framing comes from QA and test-automation study sessions y.morimoto attended, where multiple companies reportedly described the same challenge of using AI while keeping quality intact. Across those sessions, the emphasis was less on splitting roles — humans decide test viewpoints — and more on what inputs humans prepare: specifications, reference documents and structured scenarios that let AI write tests it can explain. The article lists three measures, from establishing a human-testable state first, to reviewing AI-drafted test specifications in dialogue, to making upstream documents readable by AI.

The stated shift in reasoning is that early test planning and design were traditionally justified by reducing rework costs, but with AI writing tests cheaply, the precision of the inputs given to AI is what now shapes overall test quality. Whether that holds in practice is likely to depend on how early teams can lock down viewpoints and how consistently they keep a human in the loop, and for QA and engineering leads the test is whether their specification documents are actually in a state an AI can read and reason about.

FAQ
What is an 'AI Slop' test in this article?
It refers to tests that look complete but do not actually guarantee quality — for example, tests that fix implementation details like an internal formatPrice call rather than checking the visible output such as 'コーヒー: ¥500'.
What should teams do before letting AI write tests?
Gather the materials needed for testing and build a human-testable state first. Then review and correct test specifications in dialogue with AI, and give AI access to upstream documents such as PRD, Figma and requirements.
Why does test design matter more now that AI writes tests?
Because AI lowers the cost of writing tests, the value shifts to preparing structured inputs that let AI correctly understand the specification and its intent. The article frames this as a change in reasoning, not a new idea.

AI news that matters for your work, delivered every morning.

Pick your industry and the AI tools you use, and get news related to your work every day.

Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →

Ask AI

Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.

Questions and answers are published on this page.

Next articleAilaFlow launches self-hosted Human-AI workflow platform