Gem of Gemma – Open-Source Android App Runs Gemma 4 LLM Entirely On-Device with No Internet Required After Initial Download

An open-source Android app showcases on-device AI inference using Gemma 4 and LiteRT-LM, with chat, multimodal image understanding, and phone control (22 toggleable tools including SMS, calls, alarms, flashlight, and media control) running entirely locally without cloud APIs or subscriptions.

The app downloads Gemma 4 E2B (~2.5 GB, one-time) from HuggingFace on first launch, then operates fully offline. It streams tokens in real-time, supports visible chain-of-thought reasoning via thinking mode, and uses LiteRT-LM's native function calling via ToolSet API with @Tool annotations for on-device inference.

All conversation history is saved locally on the device; no data leaves the phone. The source code is Apache 2.0 licensed, and the Gemma model is subject to the Gemma Terms of Use. Requirements are Android Studio, JDK 17+, an Android device with 4GB+ RAM, and ~3GB storage.