AgentState: Open-source proxy cuts AI agent recovery time 7,360×

AgentState is an open-source, self-hosted proxy designed to intercept LLM and tool calls from autonomous AI agents and checkpoint their execution state. When an agent fails mid-run—for example on step 87 out of 100—AgentState allows developers to pause, inspect, optionally edit, and resume execution from that exact step, rather than losing all prior context and restarting from step 0.

The tool intercepts requests at the LLM client level, supporting multiple integration paths: a native AgentStateOpenAI Python wrapper, framework-specific wrappers for LangChain and CrewAI, or a standard OpenAI client configuration that points its baseURL to the proxy at http://localhost:8080/v1. Every prompt, response, and tool invocation is automatically saved to a local SQLite database. On retry, replayed steps return results in ~15 milliseconds with zero token cost via cache hits, compared to the original LLM call cost.

In benchmark testing on complex 50-step agent runs with forced crashes, AgentState achieved a crash recovery time of ~0.015 seconds at step #45 versus ~110.4 seconds on an agent without the proxy—a 7,360× speedup. Retry token cost dropped from $2.45 per failed attempt to $0.00 (100% cache hit). The proxy also recovered from 5+ consecutive crashes without losing execution completion, versus a standard agent crashing after 1 error. Additionally, AgentState includes a human-in-the-loop (HITL) gateway that can intercept and pause sensitive tool calls (send_email, execute_command, stripe_charge) until they are approved via a dashboard or API, achieving 100% interception of unapproved actions in testing.

Additional features include multi-model fallback, which transparently reroutes requests to alternative models (e.g., gpt-3.5-turbo, Claude, Ollama) when the primary provider hits rate limits (429) or server errors (500); a fine-tuning dataset exporter that converts production agent trajectories into OpenAI-compatible .jsonl format in a single click; real-time webhook alerts to Slack or Discord when runs fail or require approval; and a session replay dashboard that lets developers visually inspect agent trajectories and rollback to any step before resuming execution. The proxy runs at http://localhost:8080/v1, with an embedded dashboard at http://localhost:8080/dashboard. Setup involves cloning the GitHub repository, activating a Python virtual environment, installing dependencies (fastapi, uvicorn, httpx, openai, playwright, Pillow, imageio), and starting the proxy server. Test demos are included to validate resilience and caching, the HITL gateway, and the full feature suite.

AgentState addresses a specific failure mode in autonomous agent deployment: mid-execution crashes that force a complete restart and token re-expenditure. The benchmark results—a 7,360× speedup in recovery time and 100% elimination of retry costs—reflect the efficiency gain from caching and checkpoint replay rather than re-running LLM calls. The human-in-the-loop approval gateway is a practical addition for production systems, preventing unmonitored execution of sensitive actions like email sends or stripe charges.

The proxy's architecture intercepts at the LLM client level, which keeps integration friction low (a baseURL swap or 1-line wrapper). By storing state in a local SQLite database rather than requiring external infrastructure, AgentState trades operational simplicity for the ability to handle complex multi-step agent trajectories. The inclusion of framework wrappers (LangChain, CrewAI) and dataset export for fine-tuning suggests the authors are positioning it as both a resilience tool and a data-collection platform for improving agent policies over time.

No comments yet. Be the first to share your thoughts!