AIToday
Large Language ModelsAI Coding AssistantsZenn AI/MLPublished: Nov 16, 2026, 16:00 JST

Claude Code PHP questions: 403 of 433 verified on PHP 8.2.0

Send on LINE
Claude Code PHP questions: 403 of 433 verified on PHP 8.2.0

3 Key Points

  1. What happened

    The author ran 433 Claude Code-written PHP questions through a custom Artisan command, questions:verify-php, on PHP 8.2.0; 403 matched the registered answers and none mismatched.

  2. Why it matters

    The zero-mismatch result covers only code output against registered answers, not whether the explanation or the other answer choices are sound, so the verification is narrower than a clean pass suggests.

Not sure about something? Ask the AI

Questions and answers are published on this page.

Summaries like this, every morning.

Context & Analysis

The setup matters as much as the result. Before generating anything, the author fixed rules: questions about code output must be executed in PHP and checked against the answer, and no question goes live until a person has reviewed it. Claude Code handled the execution checks during creation, and a separate Artisan command was built to re-run all code-containing questions at once, described in its own help text as an annual review tool. The command sorts each question first, skipping multiple-choice and web-server-dependent items, then running fill-in-the-blank choices and plain output questions through a separate PHP process with errors included in the comparison. That sorting logic is where the gap appeared: a text-prefix condition meant questions phrased as file or terminal executions never reached the comparison step, even though they asked for output. The author later widened the condition and added explicit reasons to exclude two questions that read POST body input and one that is a terminal command rather than a PHP file. Separately, re-checking the local database against the JSON turned up five missing questions and several stale answer choices, which shifted the first tally from 397 matches and 434 code-containing questions to the corrected 403 and 433. The author also notes what the command cannot see: answer choices that are conspicuously longer than the rest, and hints left in the question text, were found by reading, not by execution, and Claude Code made the fixes.

FAQ
How does the verification command check fill-in-the-blank questions?
It inserts each answer choice into the blank one at a time and runs the code, then collects which choices produce the expected output. It confirms that exactly one choice matches, so a question with two correct answers is caught.
Why were some output-based questions skipped?
The original filter only matched question text starting with a specific phrase about executing PHP code. Questions phrased as file execution or terminal commands were routed to 'not an output question' and skipped without comparison.
What was wrong with the database used for verification?
The local database was missing 5 of the 500 questions and still held pre-edit versions of several others, because the import command treats edited text or code as a new question and leaves the old version in place. The author deleted the stale versions, re-imported the JSON, and re-ran the counts.
Send on LINE

AI news that matters for your work, delivered every morning.

Pick your industry and the AI tools you use, and get news related to your work every day.

Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →

Ask AI

Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.

Questions and answers are published on this page.

Related Articles

Next articleThailand GDP grows 2.9% on AI-related exports