GitHub's Copilot code review agent initially got worse after switching to better-maintained shared code exploration tools—not because the tools were flawed, but because the instructions guiding the agent were tuned for general coding assistance rather than focused code review. After rewriting the instructions to match how human reviewers actually work (starting from the diff and asking narrow, targeted questions), the agent achieved roughly 20% lower average review cost while keeping quality the same. The lesson: agent performance depends as much on prompt design and workflow as on the quality of the underlying tools.
Summaries like this, in your inbox every morning.
Sign up free →What happened
GitHub's Copilot code review agent initially performed worse after adopting shared code exploration tools from the Copilot CLI (grep, glob, view), despite those tools being better-maintained. The team discovered the real issue was not the tools themselves but the instructions guiding the agent—they were tuned for general coding assistance, not targeted code review. After rewriting the instructions to match how humans actually review pull requests, the agent achieved roughly 20% lower average review cost while maintaining review quality.
Why it matters
Agent tools only work as well as their instructions. GitHub found that upgrading to shared infrastructure without adapting the workflows caused the agent to browse broadly and accumulate unnecessary context—wasting tokens and focus—rather than following a reviewer's focused approach of starting from the diff and asking targeted questions. For businesses using AI agents for code review or similar tasks, this shows that better tools alone don't guarantee better outcomes; the prompts and workflows around them must fit the actual job.
What to watch
The fix involved rewriting instructions to enforce a narrower workflow: start from the diff, use grep and glob for targeted discovery, batch searches before reading files, and read only exact evidence needed—avoiding broad exploration. This principle may be relevant to other agent applications where general-purpose tool instructions conflict with a specific, narrower use case.
GitHub's experience reveals a common challenge in agent development: inheriting tools designed for one context and expecting them to work in another without adjustment. The Copilot CLI's code exploration tools (grep, glob, view) were built to support interactive, multi-turn coding assistance—where a developer might ask the agent to understand a repository, plan changes, and edit files over time. That use case benefits from broad exploration and context accumulation. Copilot code review, by contrast, has a narrower scope: read a pull request diff, identify potential issues, and gather only the evidence needed to confirm or dismiss each concern.
The initial migration seemed straightforward on paper—swap one set of tool implementations for another—but the traces revealed a fundamental mismatch. The agent, following general-purpose instructions, began browsing the repository like a coding assistant would, making broad searches, guessing paths, and carrying forward extra context. This behavior increased token cost and reduced review quality because every tool result stays in the agent's context window. A human reviewer does not work this way; instead, a reviewer starts from the diff, forms specific questions ("Where is this function called?" "Is this config key used elsewhere?"), and seeks the minimal context to answer those questions.
Once the team recognized the problem was not the tools but the instructions, the fix became clear: rewrite the guidance to reflect a reviewer's actual workflow. The new instructions enforce a narrower, more disciplined pattern—use grep and glob for targeted discovery, batch cheap searches before reading, and read exact evidence only when the agent knows which file or range it needs. This shift from broad exploration to focused investigation cut average review cost by roughly 20% while maintaining quality. The lesson extends beyond code review: agent performance is shaped as much by the instructions and workflows surrounding the tools as by the tools themselves. Better infrastructure alone does not guarantee better outcomes if the guidance driving the agent's behavior does not match the actual job.
No comments yet. Be the first to share your thoughts!
Log in to join the discussion
Get curated AI news from 200+ sources delivered daily to your inbox. Free to use.
Get Started FreeFree · takes 30 seconds · unsubscribe anytime
1 minute a day. The AI essentials.
200+ sources · Email / LINE / Slack
