Hardware Safeguards to Throttle AI Performance at Runtime

The researchers framed their work around a recognition that as AI models become integral to critical computer systems—those where failures carry real consequences—the lack of control over AI intent creates a safety problem. Software safeguards, which impose behavioral constraints on model output, cannot guarantee safety against a model sufficiently intelligent to circumvent them. Hardware-level enforcement has been identified as essential, but prior proposals have been limited to coarse mechanisms like unauthorized access policies or full-chip shutdown. The authors set out to design fine-grained, dynamic intervention at the microarchitecture level. They identified the GPU memory subsystem as the intervention point and evaluated candidate "knobs"—controllable parameters—spanning capacity, bandwidth, latency, and frequency. After analysis, they narrowed to four strong candidates: L2 cache size, L2 latency, L2 bandwidth, and shared memory port access rate. Each mechanism builds on well-established microarchitectural primitives: cache way masking (selectively enabling or disabling cache ways to reduce capacity), credit-based rate limiting (controlling bandwidth), latency insertion (slowing memory access), and bank arbitration (managing access to memory banks). Evaluation showed these knobs deliver high performance sensitivity, with up to 80% performance reduction when resource availability drops to 1/8 of normal levels. Implementation overhead is negligible—less than ~10K flip flops—a tiny addition to a modern GPU. After throttling is activated, the system stabilizes in 5–80K cycles, and the controls cause minimal disruption to other parts of the chip. Multi-knob analysis revealed that combining multiple controls can amplify performance degradation beyond what each knob achieves individually, enabling control over a broader range of performance targets. The work was submitted on July 20, 2026, and represents a practical architectural contribution to the hardware safety toolbox for AI systems.

The paper addresses a fundamental asymmetry in AI safety: while software-level safeguards (behavioral constraints) have long been the standard, they rely on the assumption that an AI system will respect boundaries it can potentially circumvent if sufficiently intelligent. The authors argue that hardware-level enforcement offers a genuine last line of defense because it operates below the software layer and cannot be negotiated away by model capabilities. The gap they identify is the absence of fine-grained, dynamic intervention mechanisms—existing hardware defenses are limited to blunt instruments like full-chip shutdown or access-control policy enforcement. This work fills that gap by proposing a set of microarchitecture knobs that operate at the GPU memory subsystem level, where they can restrict resource availability without requiring additional silicon or complex new logic. The four candidate knobs (L2 size, L2 latency, L2 bandwidth, shared memory port access rate) were selected because they offer high sensitivity to performance—meaning small resource reductions yield substantial slowdowns—while remaining implementable using existing cache and bandwidth-control techniques. The fast stabilization time (5–80K cycles) and minimal collateral impact suggest these mechanisms could be integrated into production chips without disrupting normal operation for non-throttled workloads.