Agent 'done' claims checked against logs: VERIFIED 3/6(50%)

The problem the author starts from is a familiar one for anyone running agents on batches of tasks: the agent can declare a batch finished without ever checking a single tool result, and nothing outside the loop stops it. The design response is to keep the judging logic out of the agent. The Verifier compares three things — the Task spec, the agent's CompletionReport, and the ExecutionLogger's records.

The author's first attempt fed the completion report's text into the Verifier and looked for the target path in it. That was dropped because it only tested whether the agent's own sentence happened to contain the right path, which a fabricated path would satisfy just as well. Narrowing the decision to the Task spec and the execution log is what moved the system away from self-grading; reporting the log is cleared before each run so stale entries cannot trigger false matches.

The three-way split came out of a second dead end. With only verified and unverified, the wrong_target case — where the agent really does write, but to a target one character off — fell into 'no evidence', which does not describe it. Since the operational fixes differ (question the agent versus question the task spec), the outcome was split into VERIFIED, UNVERIFIED and MISMATCH. Keeping the tallies as aggregates rather than reading log lines by hand is meant to show which kinds of tasks drift most often.