Coding agents overwhelmed CI; test selection cut queue times 92%

The author encountered an unexpected scaling dynamic: one engineer with coding agents produced changes fast enough to recreate a CI throughput problem normally seen in large engineering teams. The bottleneck was not inherent to the codebase size but to the *rate* of incoming changes overwhelming a fixed verification capacity.

The root issue was contention. With one runner and multiple changes queued, each taking ~20 minutes, the queue could grow to hours while the runner cycled through work. Traditional scaling—adding runners—addresses throughput but not the underlying problem: every change still triggers the same test suite, so capacity demand grows with change rate. The author's prior experience at Shopify showed that this approach works but becomes expensive.

Test selection inverted the problem. By reducing work per change (from ~20 minutes to a couple of minutes for localized changes), the runner no longer ran near capacity. Queueing vanished, and spare capacity returned. The 92% reduction in p95 total time was almost entirely from eliminating wait, not from faster execution. This works because TypeScript's compiler provides cheap static dependency information, avoiding the runtime-tracing overhead the author had to manage at Shopify.

The approach has prerequisites: the codebase must have clear boundaries (routes, APIs, packages) that the selector can use to map changes to tests. It also requires conservatism—recording uncertainty rather than assuming safety—and nightly full runs to catch boundary-crossing bugs and refresh runtime data. Under those conditions, verification cost scales with change scope rather than change rate.