AWS sample routes coding agents via NVIDIA NeMo Switchyard on Amazon Bedrock

The sample targets a practical gap in coding agents: a single developer task can take a few model calls or dozens, and the agent keeps reading files, editing code, and running tests until the work is done. The deployment puts LiteLLM and the Switchyard plugin in one container on Amazon ECS Fargate in private subnets, with AWS WAF, a path allowlist covering /v1/chat/completions, /v1/models, and /health/liveliness, and inference traffic carried through an Amazon Bedrock Runtime interface endpoint so it stays on the AWS network. The configuration keeps the routed group and two single-model baseline groups (claude-opus-5-5 and nemotron-super-3-120b) in the same deployment, so the same stack can serve both the routed and single-model setups.

The Stage profile sets picker = "efficient_first" with confidence_threshold = 0.5 and recent_window = 3, so a new request with no tool results goes to Nemotron 3 Super and the agent must send full conversation history for Stage to read the signals. AWS notes that clients using the OpenAI Responses or Anthropic Messages paths get a 404, and that the local Docker Compose option still makes billable Amazon Bedrock calls. A plain-HTTP opt-in exists for short tests but leaves prompts, tool results, and responses unencrypted, so AWS advises rotating the key and destroying the stack afterward.