MicroLLM lab lets users benchmark tiny 135M models in their browser

MicroLLM lab arrives as a browser-based alternative to benchmark suites that judge models on writing quality. Because its checks are objective — regex or exact tokens — a 135M model that fails a check is not a problem; the failure is the measurement the tool is designed to produce. The tool separates speed, reported as tokens/s of sustained decode plus suite wall time, from accuracy, reported as a pass rate on objective tests, and it uses the latest suite per model for its charts.

The setup also allows users to write their own benchmarks in JavaScript, with the editor eval()'d in the page origin so each check runs on the model's decoded text. There is an option to prompt a larger LLM, and an estimate uses the reader's last tok/s reading if one is available.

The main constraint is that results stay on the machine, which keeps data local but also means scores are shaped by each user's browser and hardware. How useful the tool proves will likely hinge on whether its objective checks become a common reference point, and on which models users choose to put through them.