MicroLLM lab runs seven tiny AI models in your browser

MicroLLM lab's model list spans from about 26 million to 362 million parameters — sizes small enough to run inside a browser rather than on a server. The weights are stored in the browser's IndexedDB database, and Q4 quantization keeps each weight file around 15 to 216MB.

The tool relies on WebGPU, an API that lets a browser use the device's GPU for computation; where WebGPU is unavailable, it falls back to WebAssembly or JavaScript on the CPU. In a test on Firefox 157.0 for Windows, generation ran at 26.9 tokens per second and stopped mid-sentence because the default output limit is 64 tokens.

Beyond chat, the benchmark feature judges answers by whether they contain specified words or numbers and records processing time, though it does not comprehensively rate text quality. A separate Custom eval mode lets users write JavaScript conditions to check answers against their own tasks. Rob says that when developing apps with on-device AI features — such as inquiry classification or spam detection — MicroLLM lab helps compare model accuracy and speed.