LLM-budget-cap: atomic Redis spend cap for AI APIs

The developer released llm-budget-cap as an npm package designed to prevent surprise bills from uncontrolled LLM API usage. The core mechanism is straightforward: before calling an LLM API (OpenAI, Gemini, Anthropic, or any compatible service), call cap.checkAndIncrement(), which atomically checks whether the budget remains and increments the counter. If the counter exceeds the limit, the decision returns allowed: false and the application rejects the request with a 429 error; otherwise, it proceeds to make the paid API call.

The package's critical innovation is guaranteeing atomicity through a Lua script. A naive implementation—reading the counter with GET, checking the limit, then writing back the incremented value with SET—has a race condition. When two requests arrive almost simultaneously with the counter at 499 and limit at 500, both read 499 before either writes; both see 499 < 500 and both decide to proceed, so both make expensive calls even though the budget allows only one more. Under high traffic near the limit, this is not theoretical—it happens in production. The Lua script runs the increment (INCR) and TTL reset (PEXPIRE) in a single indivisible operation: current = redis.call("INCR", key); if current is 1 or the key has no TTL, set PEXPIRE; return current. Every simultaneous request receives a unique counter value, so the cap is a guarantee, not a suggestion.

The API is minimal. Import BudgetCap, instantiate with redis (an ioredis client), key (the counter name), and limit (max calls per window). The default window is 24 hours (86_400_000 milliseconds); pass windowMs to customize. Call checkAndIncrement() for a global cap, or checkAndIncrement(userId) to enforce the limit per user, tenant, or IP (the key is combined as key:userId). The method returns a BudgetCapDecision object with allowed (boolean), count (the counter after increment), remaining (limit minus count), and degraded (true if Redis failed). By default, if Redis is unreachable, the operation proceeds with allowed: true and degraded: true; this fail-open mode trades temporary unmetered usage for keeping the feature alive during a Redis outage. Setting failOpen: false makes Redis errors throw instead.

The package has zero external dependencies—it uses only ioredis (or a compatible Redis client with an eval method), which most Node.js applications already include. It works with any Node.js framework without framework-specific code. The Lua script is exported as a constant (ATOMIC_BUDGET_CAP_LUA) for auditing or reuse. The developer provides a wrapper adapter for node-redis v4, which has a different eval signature. Development includes ESLint, TypeScript type-checking, Vitest unit tests (with a real Redis instance for atomicity tests), and builds to ESM, CommonJS, and TypeScript definitions. The license is MIT.

The package solves a specific production pain point: developers shipping LLM-powered features discover at billing time that an unmetered call loop or distributed scraper has drained thousands of dollars. Basic per-IP throttling fails because an attacker can spread requests across many IPs. The standard fix—a Redis counter that resets on a time window—seems simple until you run it under load: if the increment and TTL expiration happen in separate commands, two requests arriving simultaneously near the limit both read the old counter value before either writes the new one, and both proceed even though only one should. The Lua-atomic approach closes this gap by making the INCR and PEXPIRE indivisible from Redis's perspective, guaranteeing that every call increments the counter before the next one reads it.

The package is deliberately minimal: one class, one method, no third-party dependencies beyond the ioredis client most Node.js developers already use. It also includes a practical fail-open default—if Redis becomes unavailable, the cap degrades to no limit rather than crashing the feature—though this can be toggled. The Lua script itself is exported and auditable, and the library works with any Node.js framework (Express, Fastify, NestJS) without framework-specific coupling.

No comments yet. Be the first to share your thoughts!