Orb backend: self-hosted AI that proactively messages your phone

Orb is built around a thin agent loop where the model itself decides whether to speak, act, or stay silent. The server runs on Windows (with macOS planned) and requires Python 3.11+. Installation is straightforward: clone the repo from github.com/getorb/Orb-Backend, create a virtual environment, install dependencies from requirements.txt, run a guided setup script (which requests a token and identity), and start the supervisor.py server with crash recovery. To make the backend reachable from your phone, you expose it via a tool like Tailscale (tailscale serve --bg 8340), then pair the iOS app through the menu.

The backend comprises several key components. The voice server implements a WebSocket loop: speech input runs through local Whisper (speech-to-text), and responses come back as a real conversation via neural text-to-speech—no wake word required on the phone. The agent spine is the decision-making core, with roughly 60 built-in tools covering weather, mail, files, stocks, screen access, background sessions, missions, reminders, and notifications. The multi-brain router lets you choose which model powers the conversation (Claude via your own Claude Code CLI, Grok, Codex, or local models) while maintaining a consistent persona across swaps. Between conversations, the 'mind' process runs a situation model of your life and projects, detects changes, makes predictions, and proposes work—all with approval gates so you control what actually happens. Long-running jobs (builds, research, training) can be delegated to tracked background agents (called 'sessions') that you monitor and control from your phone.

Notification delivery is treated as a critical design element: Orb uses Apple Push Notification service (APNs) with deduplication, rate-limiting (hard pacing), full-message payloads, and delivery receipts that cannot be silently dropped. For security and privacy, every variable is configurable via environment files documented in .env.example and SELF_HOSTING.md. Credentials are user-supplied: your Gmail app password for mail access, your APNs key (.p8 file) from Apple Developer for push, and your Claude Code CLI login for the conversational brain. The system is licensed under MIT (the backend only; the iOS app and branding are separate), and was built by ninjahawk. The iOS app is free on the App Store at apps.apple.com/us/app/orb-ai/id6776376035.

Orb positions itself as a direct counterpoint to cloud-hosted AI assistants by moving the intelligence and data retention entirely onto user-controlled hardware. The architecture separates concerns cleanly: the iOS app is closed-source and serves as the user interface, while the backend (where the actual work happens—voice transcription via Whisper, decision-making, tool invocation, and proactive scheduling) is fully open-source and auditable. This split allows users to trust the backend without exposing the app's code, and to customize or replace the app entirely by building their own client against the documented HTTP and WebSocket API.

The design stances—local-first data, bring-your-own credentials, structural safety (approval gates rather than discouraged practices), and treating silence as a feature—reflect a philosophy that opposes both the notification-spam defaults of many assistants and the data-collection practices of cloud platforms. By requiring users to supply their own API credentials (Claude Code CLI token, Apple APNs key, Gmail password), Orb ensures no central service holds the keys to the user's integrations. The approval-gated proposal system and comprehensive logging address a key concern in autonomous AI: that the system should never act on the world without explicit human consent, and every action should be observable.

No comments yet. Be the first to share your thoughts!