
What happened
At QA study sessions, frontend engineer y.morimoto of SonicMoove found AI wrote polished tests that fixed implementation details, sometimes checking util calls instead of the real label output — 'AI Slop' tests.
Why it matters
Because AI tests are cheap to mass-produce, the scarce skill shifts to designing what to verify first, and letting AI write tests without that design is likely to produce tests that pass but guarantee little.
What to watch
The fix hinges on human intervention — teams should get to a human-testable state first, review AI-drafted test specs in dialogue, and feed AI upstream documents like PRD, Figma and requirements.
WHO IT HITSSoftware QA engineers, test designers and frontend/backend developers who now delegate test generation to AI tools will need to own test design and document inputs themselves, rather than reviewing AI output at the end.
Summaries like this, in your inbox every morning.
The article is written from the field notes of y.morimoto, a frontend engineer at SonicMoove, who tried letting AI write tests from existing implementations and initially found the output impressive. The problem surfaced only after closer review: many tests no longer made clear what quality they were guaranteeing. The article ties this to prior writing that argues AI is strong at producing tests in volume but weak at designing where and what to guarantee, and to a second argument that AI produces tests, not trust.
The practical framing comes from QA and test-automation study sessions y.morimoto attended, where multiple companies reportedly described the same challenge of using AI while keeping quality intact. Across those sessions, the emphasis was less on splitting roles — humans decide test viewpoints — and more on what inputs humans prepare: specifications, reference documents and structured scenarios that let AI write tests it can explain. The article lists three measures, from establishing a human-testable state first, to reviewing AI-drafted test specifications in dialogue, to making upstream documents readable by AI.
The stated shift in reasoning is that early test planning and design were traditionally justified by reducing rework costs, but with AI writing tests cheaply, the precision of the inputs given to AI is what now shapes overall test quality. Whether that holds in practice is likely to depend on how early teams can lock down viewpoints and how consistently they keep a human in the loop, and for QA and engineering leads the test is whether their specification documents are actually in a state an AI can read and reason about.
Pick your industry and the AI tools you use, and get news related to your work every day.
Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →
Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.