AIToday
Large Language ModelsAI Coding AssistantsOpen-Source AIHacker NewsPublished: Jul 31, 2026, 06:00 JST6 min read

Collie: local AI coding agent that controls your browser, desktop, and code

Collie: local AI coding agent that controls your browser, desktop, and code

Key takeaway

  • Collie is a local AI coding agent that runs on your computer instead of in the cloud, giving it access to your real browser sessions, desktop, files, and code.

  • Unlike cloud agents that only read files, Collie can interact with your actual logged-in environment and verify its work by running automated tests — if an edit breaks something, the test fails and the agent tries again.

  • It works with popular AI models (DeepSeek, Claude, OpenAI, Ollama) and keeps all your data local unless you explicitly send it elsewhere.

3 Key Points

  1. What happened

    Collie, a coding agent that runs on your machine rather than in the cloud, was released. It can drive your logged-in browser, manage your desktop, record your screen, and edit code — all while keeping data local with no account or telemetry required.

  2. Why it matters

    Unlike cloud-based coding agents that only access files you hand them, Collie works with your real environment (browser session, desktop, files) and verifies its work by running it: it writes a test that must fail on broken code, makes the smallest edit to fix it, and re-runs the test to confirm. This end-to-end verification means wrong edits fail loudly instead of shipping silently.

  3. What to watch

    Collie is available now via Windows installer (one-click download) or macOS/Linux via pip; it supports multiple AI providers (DeepSeek, Claude, OpenAI, Ollama, and others); and includes a built-in benchmark lab so you can measure it against other agents on the same task — no asserted numbers, you run it yourself.

In Depth

Read the full story

Collie is a coding agent designed to run on your machine and interact with your real environment. Unlike cloud-based agents that read code from files you hand them, Collie drives your logged-in browser, manages your desktop, records your screen, takes tasks from your phone, and edits code — all on one machine under your control. Nothing leaves your computer unless you send it there; there is no account and no telemetry.

The core innovation is the verification gate. When Collie fixes something, it writes a reproduction test with an assertion (e.g., assert parse_duration("1h30m") == 5400) that must fail on broken code. It then makes the smallest edit that flips the assertion to passing, and re-runs it to confirm. The gate is armed until verification passes. This means wrong edits fail loudly instead of shipping silently as "done." The same principle scales: collie loop stops when a real shell check exits 0, and collie pack picks the best of N attempts by what actually passes.

Collie ships with several surfaces. The terminal interface (collie TUI) is the primary entry point; collie web opens a browser GUI with chat, live verification, diffs, a star-map visualization of code being worked on, and an ambient desktop (clock, weather, dock, music player, command bar). A Chrome extension lets Collie act in your real, logged-in browser — every action is fenced and CSRF-checked to prevent injection attacks. A phone remote (via iOS app) lets you tail runs or start new ones over the same Wi-Fi or anywhere through a relay. VS Code and other ACP-compatible editors (Zed, JetBrains, neovim) connect via an open protocol, not a bespoke extension.

Installation varies by platform. On Windows, you download Collie-Setup.exe and double-click it; the installer lays down a self-contained runtime (Python + Collie + semantic memory, nothing to preinstall) and opens Collie in a native desktop window. On first launch you pick a brain — an existing Claude, Codex, or Grok login is detected and connects in one click, or you paste an API key. On macOS and Linux, the core installs via pip (pip install -e ".[local,dev]" from a clone; PyPI publish is planned). The base install is tiny because the core uses only Python's standard library. Optional extras (semantic memory, rich terminal chat, keyless web search, editor protocol, Playwright for CI) are available as pip extras.

Collie supports a wide range of AI providers: mock (deterministic testing, free), Ollama (local, no key required), Anthropic (including OAuth), OpenAI, and OpenAI-compatible services (DeepSeek, Qwen, OpenRouter, Moonshot, Groq, Zhipu). Token counts are real usage — either the model's own reported count or metered via the harness for CLIs that report none. Cost and token budgets (COLLIE_MAX_COST, COLLIE_MAX_TOTAL_TOKENS) can stop a run at a ceiling.

Under the hood, Collie is built on a small, stable harness. The ContextComposer manages a lean ~1K-token fixed prefix with STABLE, CONTEXT, and VOLATILE sections. Code search works by extracting identifiers from natural-language queries and grepping the repo (ripgrep or grep), ranking files by how many of your terms each contains — no model index to go stale. The edit_file tool is exact-match and whitespace-tolerant, rejecting edits that would break Python syntax. Untrusted web and page content is fenced as data to defend against prompt injection. The implementation is one cross-platform Python codebase; OS-specific operations (killing a process tree on timeout, securing a token file, path conversion, shell choice) are isolated in harness/plat.py.

Collie includes a built-in benchmark lab. You can run it yourself on SWE-bench Verified (with Docker), Aider-Polyglot, or against other agents (Claude Code, Aider, etc.) using the same task and model. No numbers are asserted by the developers; you run the benchmarks yourself. The documentation is transparent about the levers that turn out net-neutral, not just the wins. Collie draws a personal subscription only through first-party OAuth (the same mechanism the official Claude CLI uses) and never scrapes or resells subscription tokens. The license is MIT © 2026.

Context & Analysis

Collie represents a shift in how AI coding agents approach the problem of fixing code. Traditional cloud-based agents read repositories and edit files, but operate in isolation from the actual runtime context. Collie's core differentiator is that it runs on your machine, which means it can access your logged-in browser session, interact with live websites, observe your desktop in real time, and run tests immediately to verify its work. The verification gate is the technical spine: when Collie fixes a bug, it doesn't just claim success — it writes a test that must fail on the broken code, makes the edit, and re-runs the assertion. This assert-verify loop ensures that a wrong edit triggers another repair attempt rather than shipping silently as "done."

The breadth of capabilities (desktop control, browser automation, screen recording, phone remote, editor integration) exists because Collie's agent built them all, according to the documentation. This is significant: the features are not bolted-on conveniences but demonstrate the harness's strength. A desktop app and iOS companion built by the same reasoning engine that fixes code suggests the underlying system is robust enough to handle real-world tasks end-to-end.

Collie also addresses privacy and control: all data stays on your machine unless you explicitly send it elsewhere. There is no account, no telemetry, and the default is local models or cheap API keys rather than locked-in subscription services. The architecture is intentionally simple — a small, honest harness with clear seams (ModelProvider, ToolRegistry, EmbeddingProvider, SqliteMemory) so each part can be swapped without redesigning the whole system. A built-in benchmark lab lets users measure Collie against other agents on the same tasks, and the documentation is transparent about what does and does not work.

FAQ

How does Collie verify that a fix actually works?
Collie writes a reproduction test with an assert that must fail on broken code, makes the smallest edit to fix it, and re-runs the test. A run is not marked done until the test passes; a plausible-but-wrong edit fails loudly and triggers another repair attempt.
What do I need to install Collie?
On Windows, download and double-click Collie-Setup.exe from the latest release. On macOS or Linux, use pip (pip install -e ".[local,dev]" from a clone; PyPI publish is planned) and run collie setup to pick a provider.
Can Collie work without paying for an API key?
Yes. The core has zero third-party dependencies, and you can run mock (free deterministic testing), Ollama (local, no key), or other local models. Cheap API keys (DeepSeek, Qwen, OpenRouter, etc.) are the default if you choose a commercial provider.

Get the latest Large Language Models news every morning

AI-summarized, only the topics you pick — one digest a day via Email, Slack, or Discord.

Free · takes 30 seconds · unsubscribe anytime

Related Articles

Next articleEngineer trains LSTM to mimic human mouse movements

The AI news that matters, in one minute each morning.

Sign up free