Zenn author: cheap AI failures reshape prototyping

The essay is framed as a follow-up. An earlier piece covered keeping specifications, implementation and running state aligned; this one asks how, on top of that base, you combine probabilistic search for candidates with checks run under fixed conditions. The author's analogy is subscription streaming versus buying a single record or cinema ticket: with a flat fee, a wrong pick costs time but not another purchase, which lowers the barrier to trying something else.

Much of the argument sits with what must not be relaxed. The author cites a piece by Hiro Hasegawa about how Silicon Valley has moved past talking about AI while Japan has distributed it and stopped, and references Hugging Face material on generation methods to make the point that using a probability model does not mean getting a different answer every time. A separate section warns that if AI writes both the implementation and the tests, the same misunderstanding can enter both, so expected values should come from verified requirements or current specifications rather than from the generated answer itself. It also raises what the author calls a "knows but won't say" problem, distinct from hallucination, and points to paltering as treated in language-model research, including a 2025 paper by Liang et al., while noting that research does not prove the intent behind any response they saw.

The stakes turn on whether iteration can be stopped at the right moment. The author notes that checking tool results cut off at a limit means the returned count is not the total, so a correct "two matches" can still hide unexamined cases, and that current tools do not all return truncation, failure and no-match information in a common format. If review is dropped to save tokens and rework grows, the overall job may not get lighter — which suggests the design of trial-and-judgment, not raw attempt counts, is likely to decide whether cheap failure pays off.