LLM API gateway masks real keys, adds spending controls

The gateway is a Cloudflare Worker that intercepts API requests from clients and routes them to DeepSeek, OpenAI, or Anthropic on behalf of the user, keeping the real API credentials hidden from the client side.

Clients authenticate using opaque tokens created by an administrator and stored in a D1 database. Each token is scoped to exactly one provider and enforces three constraints: an expiration time (stored as milliseconds since Unix epoch), an IP allowlist (checked against Cloudflare's CF-Connecting-IP header), and a provisioned budget in USD. On each incoming request, the gateway looks up the token, verifies it has not expired, originates from an allowed IP, and has not yet spent its full budget. If all checks pass, the gateway substitutes the real upstream API key (stored in the Worker's secrets) and forwards the request. If any check fails, the request is rejected.

The implementation supports five API endpoints across the three providers, verified by @er91 on 2026-07-14: DeepSeek's /deepseek/chat/completions and /deepseek/v1/messages endpoints; OpenAI's /openai/v1/responses endpoint; and Anthropic's /anthropic/v1/messages endpoint. The gateway works with agentic coding tools including OpenCode, Claude Code CLI, and Codex CLI.

Deployment requires creating a new Cloudflare Worker, setting up a D1 database using the provided schema, and installing the Rust toolchain (to build the wasm32-unknown-unknown target). Administrators configure upstream API keys via the Worker's secrets, then issue test tokens by inserting rows into the tokens table. For example, a token might allow spend up to $1.00 USD, with expiration at a specified Unix timestamp, and restricted to a single client IP or opened to all IPs using ["*"].

The gateway has two documented limitations. First, budget enforcement is not strict: the gateway only checks if a token has exceeded its budget at the beginning of a request, so a single large request or multiple concurrent requests can go over budget before being rejected. Second, cost tracking is not fully reliable: if the client or upstream connection fails abruptly, or if the gateway crashes while calculating or persisting the cost, the request is still served but may not be billed. The author notes these gaps and suggests enhancements (such as setting max_tokens dynamically based on available budget, or implementing monitoring to detect billing failures) but has chosen not to implement them in the initial version to keep the gateway simple and to avoid modifying requests.

The gateway addresses a practical security concern: teams distributing API access to multiple developers or tools face a dilemma between convenience and credential safety. Sharing real API keys widely increases the risk of leaks and makes it difficult to audit or limit spending per user or application. By introducing a token-based layer, the gateway decouples the problem of credential distribution from the problem of spending oversight.

The design scope is narrow and intentional. The gateway supports only static credentials (API keys and long-lived access tokens), not refresh-token flows, because implementing token refresh within the gateway would require either access to unstable internal APIs or a way to reliably identify users across changing tokens—neither of which is feasible for all three providers. The gateway is built for Cloudflare Workers and D1 database, which means deployment is tightly coupled to Cloudflare's infrastructure.

The two documented limitations—non-strict budget enforcement and unreliable cost tracking during failures—represent conscious trade-offs. The author deliberately avoids modifying requests (e.g., clamping max_tokens) and does not implement full monitoring and retry logic, preferring a simpler gateway that forwards requests as-is. Users relying on strict spend limits or perfect billing accuracy would need to add external monitoring or implement request modification themselves.

No comments yet. Be the first to share your thoughts!