Claude Code: written completion criteria add 3–5 tests vs 1 per run

The author set strict conditions to keep the comparison clean. All runs started from the same commit (e161664) as the publicly released v1.2.2, restored with git reset --hard and git clean -fd app before each run, with the equivalent of CLAUDE.md left at its original 18 lines and only the instruction text changing between the one-line and criteria versions. Run order was A1 through B3 in sequence, with manual checks inserted after each, on Claude Code 2.1.281 with Claude Opus 5.5 (claude-opus-5-5[1m]) on 2026-09-26. Because the build cache carried across runs, the reported durations of 55–87 seconds for A and 59–100 seconds for B cannot be used to compare speed.

The divergence appeared in areas the compiler and the project's existing rules did not cover. The one-line runs each added a single test confirming that dates inside the current month matched and adjacent months did not; in one of those runs the month end was verified at 5/31 00:00, which would also pass an implementation that stopped at midnight. The criteria runs instead tested the boundaries as written, and one report explained that the boundary value was computed separately as "midnight on the first of the next month minus 1 millisecond" rather than reusing the same dayEnd function as the implementation, so a faulty implementation could not make the test pass. That separation was not spelled out in the criteria, and the third criteria run did reuse dayEnd in its tests. Ordering tests were newly created in PeriodPresetsTest in all three criteria runs and absent in all three one-line runs.

A separate trial in the criteria series had the model output a plan first using --permission-mode plan, then resumed the session with --resume to ask it to proceed. The plan listed the five files to change and noted that Period branching occurs in only one place in Period.kt and Kind branching only in labelOf() in PeriodSelectScreen.kt. It also included a test not written into the criteria, covering the case where "now" falls exactly on the first or last day of the month; that run ended up with the most tests of any run, at 6. The author also warns against taking the model's reports at face value: one report attributed an UP-TO-DATE analysis to an edit hook that built first, but the logs show no hook ran, and the hook from the previous article only formats with ktlint, while the UP-TO-DATE result came from build output left in app/build by the previous run.