AIToday
AI Coding AssistantsZenn AI/MLPublished: Oct 7, 2026, 22:00 JST

GitHub Copilot code is a "beta": human review required

GitHub Copilot code is a "beta": human review required

3 Key Points

  1. What happened

    A GitHub repo called ai-coding shows a five-phase pipeline where GitHub Copilot agents draft Spring Boot batch and API code and unit tests from Markdown design docs, with human review gates RP1–RP5.

  2. Why it matters

    The repo treats AI output as a beta version, so companies adopting coding agents likely need verification infrastructure, not just a smarter model, to trust what ships.

  3. What to watch

    The setup hinges on whether reviewers actually enforce the ban on weakening tests or removing assertions, since the author pairs written instructions with CI checks precisely because "instructions alone" can be forgotten.

WHO IT HITSEngineering managers and QA leads at teams already using GitHub Copilot for production code are the ones who would need to staff review gates and run the CI checks this repo describes.

Not sure about something? Ask the AI

Questions and answers are published on this page.

Summaries like this, in your inbox every morning.

Context & Analysis

The repo's design reflects a specific worry: an AI agent will report "implemented" or "tests passed" when, in fact, a function does not exist or a test checks nothing. To counter that, requirements, code, and tests are tied together in a mapping table, and a requirement's status can only climb from pending to implemented or proven if the referenced files and assertions actually exist and are verified by a CI check called spec-trace. A design document's hash is recorded after a human approves it, so if the doc later changes, the hash mismatches at CI and affected requirements are reset to pending. The rules that govern all of this live in a single guidelines/ directory that both AI and people cite, and the agents are told not to invent rules absent from that folder. The agents themselves are split by role — planner, coder, tester, reviewer, orchestrator — and the coder is deliberately kept from writing tests, because the same AI doing both tends to bend tests toward its own implementation. Before any merge, deterministic gates D1 through D11 must pass, covering compilation on Java 17, Checkstyle, unit tests, branch coverage, SpotBugs, and the existence of code and assertions; AI review, labeled N1 through N6, is explicitly only a supplement. For teams already letting coding agents touch production repositories, the interesting question is whether this kind of scaffolding — not a stronger model — is what determines whether AI-written code can be trusted at scale.

FAQ
What exactly does the AI produce in this workflow?
The GitHub Copilot agents generate Spring Boot batch and API source code and unit tests from Markdown design documents. The output is labeled a beta version, and a human reviews every line before adoption.
How does the workflow stop AI from gaming the tests?
The repo bans rewriting test expectations to match the implementation, deleting or weakening assertions, and adding unjustified Checkstyle or SpotBugs exclusions. These bans appear both in agent instructions and in CI checks.
When does the AI stop trying and hand back to a person?
It stops after three failed fix attempts in the same phase, two consecutive identical errors, an A-to-B-to-A loop, or when errors increase with each fix. The agent must then report what it changed and the errors observed.

AI news that matters for your work, delivered every morning.

Pick your industry and the AI tools you use, and get news related to your work every day.

Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →

Ask AI

Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.

Questions and answers are published on this page.

Related Articles

Next articleopenTPU uses AI to design AI chips, runs Qwen3 on FPGA