AWS shows how to blend SageMaker and Bedrock models in one agent workflow

AWS published a technical walkthrough showing how to build multi-agent workflows that blend models from Amazon SageMaker AI and Amazon Bedrock under a single Amazon Bedrock AgentCore runtime. The example system uses three agents: an orchestrator (Claude Haiku 4.5 on Bedrock) that classifies user intent and routes tasks, a budget agent (Claude Sonnet 4.6 on Bedrock) that handles 50/30/20 budget breakdowns using structured Pydantic output, and a financial analysis agent (Qwen 3.5 9B on SageMaker AI) that performs stock analysis and portfolio construction. User requests enter the orchestrator, which uses the agents-as-tools pattern from Strands Agents to route to either the budget or financial analysis agent; both call their respective models and return results back through the orchestrator.

The architecture combines three model-hosting paths: the orchestrator and budget agent invoke models through Amazon Bedrock, while the financial analysis agent invokes Qwen 3.5 9B through a SageMaker AI real-time endpoint using an OpenAI-compatible API. The post walks through deployment of Qwen 3.5 9B using the vLLM Deep Learning Container image vllm:0.22.1-gpu-py312-cu130 on ml.g6e.2xlarge (1× L40S GPU with 48GB VRAM), then shows how to integrate it into Strands Agents and ship the entire workflow to Amazon Bedrock AgentCore runtime using the bedrock-agentcore-starter-toolkit.

A critical implementation detail is observability. Amazon Bedrock AgentCore runtime automatically instruments agents using OpenTelemetry, but the default instrumentation has a blind spot: it recognizes Amazon Bedrock model calls (made through boto3) as generative AI operations and captures token counts automatically, but it does not recognize Amazon SageMaker OpenAI-compatible endpoints as generative AI calls. This means tokens consumed by the Qwen model are completely invisible in traces, making cost monitoring, regression detection, and latency debugging impossible. The root cause is that Strands' OpenTelemetry integration emits spans for tool calls and agent lifecycle events, but it doesn't emit gen_ai.chat spans with token attributes for the OpenAIModel provider. The solution is to manually emit a custom gen_ai.chat span that wraps the SageMaker agent invocation and extracts token usage from Strands' internal AgentResult.metrics.accumulated_usage dictionary. The keys are inputTokens, outputTokens, and totalTokens. A critical prerequisite is adding stream_options: {"include_usage": true} to the OpenAIModel parameters, because by default vLLM doesn't include a usage chunk in streaming responses—Strands receives text chunks but never a final usage object, so accumulated_usage stays at zero. With this parameter, vLLM sends an extra final chunk with token counts. The post includes example trace output showing a gen_ai.chat span with 1391 input tokens, 1432 output tokens, and 2823 total tokens, with a duration of 37237386894 nanoseconds, demonstrating token-level visibility for the SageMaker-hosted Qwen model alongside automatically instrumented Bedrock AgentCore spans.