AIToday
AI Coding AssistantsZenn AI/MLPublished: Oct 23, 2026, 13:00 JST

Zenn guide: five checks catch AI code that runs but is silently wrong

Send on LINE
Zenn guide: five checks catch AI code that runs but is silently wrong

3 Key Points

  1. What happened

    A Zenn article lays out five checks non-engineers can run themselves: decide the expected answer first, test boundary values, test empty and single-item inputs, suspect data types, and count records in versus out. It shows an AI-written discount function returning 990 instead of 900 with no error.

  2. Why it matters

    The piece argues the dangerous AI code mistakes are the ones that don't stop, so readers are told to demand execution output rather than ask whether the code looks right — an approach it says never requires reading the code.

Not sure about something? Ask the AI

Questions and answers are published on this page.

Summaries like this, every morning.

Context & Analysis

The starting point is a specific kind of unease: the code ran, no error appeared, but the writer cannot tell whether the result is right. It separates mistakes into two kinds — ones that stop the program, which are low-risk because they force attention, and ones that keep running with a quietly different result, which it calls the high-risk kind and the one AI-written code tends to produce. Its example is a discount function where 100 and 1000 were swapped, returning 990 yen instead of 900 with nothing to flag it.

The five checks are framed as process, not expertise. Deciding the expected number before asking, probing the exact boundary and its neighbours, checking what happens with zero or one item, watching whether digit counts reveal text being treated as numbers, and comparing counts in against counts out are all things a reader can do without reading the code. These confirm only the cases tried, not overall correctness, and some decisions — such as whether zero records should show a dash, zero, or an error — belong to the person specifying the work, not the AI.

Its closing argument is about how to ask. Requests for an opinion, it says, get a reply either way, so the stronger form is to ask for actual run output, since results cannot be invented the way opinions can.

FAQ
What does the discount example show?
A function meant to take 10% off 1000 yen returns 990 instead of 900 because 100 was mistyped as 1000. No error or warning appears, so the wrong number would reach a customer unless the expected answer was decided in advance.
Why test boundary values?
An adult check was written with > 20 instead of >= 20. It looks correct at 21 and 19 but fails at exactly 20, so it says to test the exact value and the ones on either side.
What is the cheapest check?
Counting records. When rows are grouped by name, two entries for the same person collapse into one, so 3 rows produce 2 results and the total falls from 6000 to 5000. Displaying counts in and out exposes it without reading code.
Send on LINE

AI news that matters for your work, delivered every morning.

Pick your industry and the AI tools you use, and get news related to your work every day.

Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →

Ask AI

Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.

Questions and answers are published on this page.

Related Articles

Next articleAgentic AI Foundation adds four projects, hits 10 total