
What happened
On the same gemini-3.8-flash model, the prompt-only agent hit 20/20 functional tests but broke the 65-line limit with a 74-line tariff.py, while the hook-gated agent passed 5/5 rules and 20/20 tests in one call.
Why it matters
The prompt-only agent's 74-line tariff.py slipped through because only a functional smoke test ran at submit, so the rule breach reached the repository unnoticed — evidence that soft prompt rules alone cannot guarantee compliance.
Summaries like this, every morning.
The experiment targets a common practice in AI coding agent operations: writing architecture conventions — no cross-layer dependencies, no mutable globals, mandatory return type annotations, a maximum line count per file — as dozens of lines of instructions in CLAUDE.md, .cursorrules, or system prompts. This Prompt-Only approach has a known failure mode, because the model's attention is consumed by implementing functional requirements and soft constraints are probabilistically skipped, especially when a smoke test passes and nothing flags the breach. Mode A demonstrated exactly that: the agent followed local syntactic rules like banning imports and using tuples, but expanded tariff.py to 74 lines, nine over the limit, and submitted confidently because only a functional smoke test ran at commit.
Mode B attacked the problem in two stages. At session start, a deterministic AST audit listed the 14 violations in the initial fixture and told the agent that all five rules, including MAX_RULE5_LINES = 65, would be mechanically blocked at COMMIT_AND_SUBMIT. That hard gate let the agent plan line budgets from Turn 1, compressing intake.py to 35 lines and tariff.py to 60. The result was full compliance without a single retry, with lower output tokens, shorter elapsed time, and lower estimated API cost than Mode A. The practical guidance is to move any deterministically checkable rule into an AST linter or pre-commit hook, inject the initial violation list into the prompt, and reject commits with line-numbered errors — a pattern framed as preventing architecture drift.
Pick your industry and the AI tools you use, and get news related to your work every day.
Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →
Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.
The author fed the 'desired candidate profile' text from 145 job postings to Claude, turned the extracted labe…

The FAA halted all departing US flights on Friday, three days after its SMART AI tool reached all 22 regional…

Researchers de-duplicating leak-site data say alleged ransomware attacks reached nearly 3,000 over the summer…

Jamie Dimon said JPMorgan Chase is "not gonna allow just any" outside AI agent inside the company, warning "Yo…

OpenAI is dissolving its standalone robotics team and folding researchers into the main research team under Ch…

Jacob Coxon, a former OpenAI and Anthropic researcher, cited an Anthropic IPO roadshow slide showing pre-train…
