28.9M-parameter LLM runs on $8 ESP32-S3 microcontroller

The achievement represents a shift in how language models can be deployed at the hardware extreme. Traditionally, fitting models onto microcontrollers required severe parameter reduction because the entire model had to fit into fast memory—the ESP32-S3's 512KB SRAM—which is why prior models on this chip class maxed out at 260 thousand parameters. By adopting Google's Per-Layer Embeddings architecture, the developer decoupled the model's lookup tables (embeddings) from its reasoning core, allowing 25 million of the 28.9 million parameters to live in the chip's slower 16MB flash memory. Only the active computation—the "thinking" part—needs to reside in fast memory, fundamentally changing the memory constraint equation.

The model achieves this while maintaining practical performance: 9 tokens per second is sufficient for real-time text generation on a screen, and coherent short-story generation on TinyStories proves the architectural trade-off does not destroy reasoning quality at scale. The deliberate choice to train on TinyStories acknowledges the model's limitations (no question-answering, instruction-following, or factual knowledge), anchoring expectations to what the architecture can actually deliver. This is explicitly about demonstrating that large models can run on tiny edge hardware, not about competing with general-purpose AI systems.