Local AI models fail at basic math — here's why your chatbot needs code execution, not just language prediction

A developer tested Qwen 2.5 Coder (an open-source AI model runnable on personal hardware) to sum 23 stock transactions. The model gave seven different wrong answers—ranging from 859 to 2,364 shares—before finally getting 1,884 correct. The model confidently produced plausible-looking numbers without actually computing, because language models predict the next token rather than execute arithmetic.

The real problem was not the model itself but the missing code execution layer. When the developer added a code interpreter (a tool that lets the AI write Python, then actually runs that code), the model wrote correct addition scripts and got the right answer instantly. ChatGPT and Claude.ai work reliably on math because they have built-in code execution; bare open-source LLMs do not.

If you're setting up a local AI for your business—whether for data analysis, financial summaries, or any calculation-heavy task—a simple chat interface talking to Ollama (an open-source model manager) will give you confident-sounding garbage. You need a harness (tools like Open WebUI with Code Interpreter, or Cline) that can execute real code and show you the results. Without it, you're getting mental math from a token predictor, not reliable computation.