Parsimony benchmark scores AI models on code bloat

The benchmark arrives at a moment when AI coding tools are commonly judged by whether they solve a problem and what that costs. Parsimony adds a third axis: how much code the model writes to get there. Its unit of measurement is designed to resist gaming — counting conditional branches, function calls, variable names and numbers rather than lines, and excluding line breaks, whitespace and comments so that compressing code onto fewer lines does not flatter the result. Test code, documentation and automatically generated files are left out of the measurement.

The published comparison places the Intelligence Index, calculated by the AI evaluation site Artificial Analysis, on the vertical axis and added code relative to other models on the horizontal axis. The spread suggests that a higher index does not by itself predict leaner fixes. The developer cautions that this cuts both ways: shorter code can be harder to follow, and volume alone says nothing definitive about readability or design.

Because the source code and measurement data are public on GitHub, the results can be reproduced, and the developer is soliciting fix data for additional models and feedback on the evaluation method — a sign the comparison set may expand beyond the models shown so far.