Sawtooth Memory introduces asynchronous hierarchical memory framework for LLM agents to eliminate blocking latency and data loss in conversation summarization.

Sawtooth Memory moves LLM summarization from the main application thread to an asynchronous background worker, allowing the system to instantly store user messages and return control in milliseconds instead of freezing the application for 5-10 seconds during summarization.

The system uses a four-layer hierarchical memory stack (L0 System, L1 Working, L1.5 Entities, L2 Archive) with an immutable ledger to prevent hallucinations—critical facts like UUIDs and names are extracted before summarization and guaranteed to be retained with 100% recall accuracy.

In a local GPU benchmark on an NVIDIA RTX 5060 running phi4-mini on a 20-message conversation, Sawtooth achieved 11.3× faster main thread latency (5.70 seconds versus 64.15 seconds) and 10% lower token cost (454 tokens versus 506 tokens) compared to standard summary memory.

The project is available for installation via pip and provides integrations with Ollama for local models and cloud APIs (OpenAI, Anthropic, Google), plus native support for LangGraph as a checkpointer replacement.