Solène's braindead-local-intelligence runs local AI on AMD

Solène built the project for a specific need: experimenting with local AI without working through long guides or picking from thousands of model files. The setup bundles a chat LLM server compatible with the OpenAI API, an image generation server, SearXNG for web search and Open WebUI as the interface, all behind a single address that can be protected by a token if exposed externally.

The design choice Solène highlights is model swapping. Rather than choosing between the LLM and the image model, only one is loaded in memory at a time, loaded when a request arrives and unloaded after a few minutes without traffic so GPU memory frees up. Solène's own machine, built in 2021 with 32 GB of RAM and a Radeon RX 6700 XT, still works well for this.

Solène's expectations are modest. In practice, Qwen3.6 35B-A3B with web search gave good results, but answers can take minutes because half the model runs in RAM, and the computer gets hot and noisy. Solène notes that even spending several tens of thousands of euros or dollars would not approach service-based LLMs, which use more advanced models and much larger context windows. The pitch instead is privacy and play: nothing leaves the computer except the web searches.