Researcher's early KV cache quantization benchmarks show no performance difference between f16 and q8 levels on SWE-bench-lite tasks.

Early results from SWE-bench-lite testing across different KV cache quantization levels show no visible performance gap between f16 and q8

Results for other quantization levels appear inconsistent with random variance between runs, requiring more complete benchmark data across full model sets

Researcher raises concern that SWE-bench may suffer from training data contamination, as models could have been trained on benchmark data, affecting reliability

Public dashboard, repository, and results dataset are available on Hugging Face and GitHub for transparency and reproducibility