AWS adds rate limiting to Bedrock AgentCore gateway

Bedrock AgentCore gateway addresses a fundamental challenge in managed AI infrastructure: ensuring fair resource allocation when multiple callers (users, teams, or applications) share a single entry point. Prior to this update, organizations had no built-in mechanism to prevent a single heavy user from degrading service for others or to enforce different policies across user tiers. Rate limiting closes that gap by enabling fine-grained, identity-aware traffic shaping at the gateway layer.

The two-layer enforcement model—combining per-group and per-user limits—reflects a practical requirement. Group-level limits alone would allow dominant individuals to consume an entire group's quota, while per-user limits alone would not prevent one user group from starving another. By evaluating both independently with AND logic, the system protects both cross-group and within-group fairness. The use of JWT claims and IAM principals as dimension keys integrates seamlessly with existing identity providers (such as Microsoft Entra ID in the example) and role-based access control policies already enforced by Bedrock AgentCore, creating a cohesive identity and authorization story.

The distinction between token-based rate limits (for inference) and request-based limits (for all targets) reflects the different cost profiles of AI workloads: a streaming inference call that returns thousands of tokens consumes one request slot but many token slots, so both metrics are necessary. Service-managed quotas provide an account-level safety ceiling, ensuring no customer-defined policy can overwhelm the shared AWS infrastructure.