Developer launches text-to-SQL benchmark tool at sql-benchmark.nicklothian.com to compare local and OpenRouter AI models with surprising results.

Created an agentic text-to-SQL benchmark that converts English queries into executable SQL with debugging capabilities across multiple rounds

Benchmark consists of 25 questions and runs in under 5 minutes for most models, designed to be quick while effectively differentiating top performers

Added community-requested models to the test suite and enabled users to run the benchmark against their own servers via a WASM version

The agent can view query results and iteratively modify SQL to fix issues, though with constraints on the number of debugging attempts allowed