GitHub Copilot matches rivals on fewer tokens

Key takeaway

• GitHub released benchmark data showing its AI agent harness, which powers GitHub Copilot tools, matches the performance of competing model-vendor harnesses on software engineering tasks while using fewer tokens.

• The harness supports multiple AI models from different providers—GPT, Claude, Gemini, and others—giving developers flexibility to choose the best model for each job without sacrificing quality or efficiency.

3 Key Points

FAQ

Which models does GitHub Copilot's harness support?

The harness supports 20+ frontier models across the GPT, Claude, Gemini, and MAI families, plus bring your own key for open-source and local models.

How does GitHub Copilot's performance compare to Claude Code and Codex CLI?

Task resolution rates are on par with model-vendor harnesses when using the same model and benchmark task, while showing lower token consumption across most configurations. On TerminalBench, GitHub Copilot's markers sit within overlapping variance ellipses with same-model competitors on both task completion and cost per task.

What unique capability does GitHub Copilot's multi-model architecture enable?

The multi-model design unlocks harness-level capabilities a model-vendor harness cannot offer—for example, Rubber Duck uses cross-model-family critique, where one model reviews another's work to improve outcomes beyond what any single model produces alone.