
What happened
The developer reproduced false passes locally with Python 3.14 and pytest 9.1 on Windows Git Bash: `pytest tests; echo "EXIT=$?"` showed EXIT=1 yet the overall exit code was 0, and piping to `tail -n 1` also gave 0.
Why it matters
Because the overall exit code belongs to the last command in the chain, this pattern makes a failing test suite look successful — the same signal Claude Code reads to report back that tests passed.
Summaries like this, in your inbox every morning.
The developer's setup is a single person building business software for small operators and handing development to Claude Code, a coding agent, with test results arriving in the AI's own words. The failure mode he hit was not a wrong test, but a wrong shell pattern. When he wanted to see the exit code he appended `echo "EXIT=$?"`, and when he wanted to trim output he piped it to `tail -n 1`. Both patterns let the last command in the chain own the overall exit code, so a pytest run that actually failed delivered a success signal to the agent, which then reported success. He reproduced the effect with Python 3.14 and pytest 9.1 on Windows Git Bash, and found that adding `set -o pipefail` before the pipe restores the failure signal, though he considers not transforming the output at all the safer route. Two other cases sit outside the success/failure split he had been using. `pytest` returns exit code 5 when it finds no tests, and the developer worries an AI reading only screen text would see no "failed" and call it clean. And a command that cannot start at all, such as npm launched from Python on Windows failing with FileNotFoundError, is neither a pass nor a fail. His rules therefore make reports carry the command and its exit code, treat a run that never started as a third category, and refuse to count a result taken before the code was changed.
Pick your industry and the AI tools you use, and get news related to your work every day.
Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →
Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.
Imago Platform's first client, a lighting and electrical business, ingested a large collection of PDFs, but it…
Google brought Gemini 4 Argon to Gemini app Pro and Ultra subscribers, two weeks after it went first to vetted…
GreenCore Solutions Corp

Simon Willison added Gemini 4 Argon support to llm-gemini, his Gemini API tool

Anthropic built Scion, its own source control system to replace Git, completed the migration in five months wi…

A team leader says AI-driven development has not reduced his workload
