LLM-Shield-Proxy: Zero-Egress PII Redaction in 55MB RAM

The problem LLM-Shield-Proxy addresses is a fundamental tension in enterprise AI adoption: organizations sending requests to external LLM providers (OpenAI, Anthropic, Google, DeepSeek, Mistral) must comply with SOC 2, HIPAA, and GDPR regulations that prohibit unmasked PII egress, yet the existing open-source solutions for local PII detection—chiefly Microsoft Presidio and spaCy—carry heavy memory and latency penalties (1GB+ RAM, 50–150ms per request) that are incompatible with real-time streaming chat interfaces.

LLM-Shield-Proxy reduces this to a solved problem by operating a three-tier detection architecture entirely within a corporate VPC. Tier 1 uses regex to catch structured PII (SSNs, email addresses, IP/MAC addresses) and high-entropy secrets (API keys, hex tokens) with deterministic accuracy. Tier 2 applies Shannon entropy scoring to identify cryptographic tokens by statistical signature alone. For organizations requiring higher recall on conversational PII (patient names, organization names buried in free text), Tier 3 optionally layers in a quantized BERT-NER model via ONNX runtime, claiming >95% F1 recall on benchmark datasets while staying under 65MB additional RAM—roughly one-tenth the footprint of Presidio or spaCy.

The architectural innovation is that the proxy operates as a drop-in reverse proxy requiring zero code changes, no external API calls, no persistent storage, and submicrosecond latency overhead. Because it natively implements the OpenAI API schema, an existing application routing requests to OpenAI need only change its endpoint URL; the proxy transparently intercepts, redacts deterministically, re-synthesizes streaming packets (using either realistic placeholder names or bracketed tags), and forwards the masked payload upstream. This "zero-egress" model keeps sensitive prompts and responses within the corporate boundary at all times.