livenerf benchmark tracks whether Claude Opus 5.5 degrades after release

Complaints that an AI model has gotten worse after launch are common, but they can stem from several things: quantization to cut running costs, shrinking the model, lower thinking effort, or routing changes. They can also be pure perception, where experienced users start noticing only the bad answers. Livenerf tries to separate those cases by collecting clean test data continuously from a model's release date, so the question of whether a model truly degraded can be argued with evidence rather than impressions.

The benchmark is built on Inspect, the open-source evaluation framework published by Britain's AI Security Institute, and it deliberately avoids questions a model always gets right or always gets wrong. Instead, it watches the ones a model sometimes gets right and sometimes wrong, where a real drop should be easier to see. For Claude Opus 5.5, that means 78 questions sitting near 50% accuracy.

The current read is that Claude Opus 5.5's accuracy moved up and down day to day from September 24 to 30 but stayed within a band. Livenerf's method is to set a baseline from the first 10 days after release and then track the next 10-day average, so a clearer verdict depends on that comparison playing out. One caveat is that livenerf runs on the subscription version of Claude Opus 5.5 rather than the raw API model, and some biology and math questions were refused by safeguards and left blank, which may limit how completely the results reflect what developers actually use.