GitHub Copilot benchmarks now includes Terminal-Bench-Science 0.1 and adds a new view that compares agent harnesses, both available in public preview. The site compares models in the GitHub Copilot agent harness, or harnesses using the same model, across established software engineering benchmarks, weighing resolution rate, reasoning effort, and cost per task.

Terminal-Bench-Science 0.1

In addition to software engineering benchmarks, the site now includes Terminal-Bench-Science 0.1, which evaluates AI agents on research workflows across scientific domains. This extends coverage beyond writing code to challenging workflows drawn from scientific research. Tasks are authored and reviewed by domain experts across the life, physical, earth, mathematical, and engineering sciences. Results are available via the benchmark picker.

New harness view

The new Harnesses view compares GitHub Copilot CLI with Claude Code and Codex CLI using the same model at the same reasoning effort.

Harnesses view on GitHub Copilot benchmarks comparing resolution rate against average cost per task for Copilot CLI, Claude Code, and Codex CLI on DeepSWE 1.1

The view includes:

  • Scatter plot: Each harness result is plotted by resolution rate against mean cost per task. Lines connect matched results, and shaded ellipses show the run-to-run variation observed across published runs.
  • Paired differences: For each matched model, Copilot CLI’s resolution advantage (in percentage points) and cost savings (as a percentage) are shown side by side, with approximate 95% bootstrap intervals.

Only exact model pairs at medium reasoning effort are shown. Copilot-only configurations, like GitHub Copilot Fusion, and unmatched results are excluded rather than compared against a different model.

Much of the work we do to make Copilot better happens at the harness-level, and this view makes those improvements visible.

Explore the results on the GitHub Copilot benchmarks site.