Smartsheet launches AI agent interface on AWS; cuts token use 35–47%

Smartsheet, an enterprise work management platform relied on by hundreds of thousands of organizations, recognized that as enterprise teams adopt AI agents, those agents need structured access to internal data—but most systems are not built for that. To bridge this gap, Smartsheet built a remote Model Context Protocol (MCP) server on AWS that gives AI clients direct access to Smartsheet's data and capabilities.

AI assistants like Amazon Quick and Claude Desktop help users interact with Smartsheet through natural language—analyzing project data, updating tasks, creating sheets, managing workspaces, and more. Beyond assisted use, enterprises are building custom AI agents that work autonomously without human prompting. These agents can coordinate through Smartsheet using MCP to capture requirements, pick up tasks, attach test results, and draft documentation—all in the same sheets human counterparts use. Smartsheet reports that these optimized workflows compress work that previously took weeks into days or hours.

The architecture is built on AWS infrastructure. The MCP server runs on AWS Fargate for Amazon ECS (Elastic Container Service), behind an API gateway layer that includes AWS WAF (Web Application Firewall), AWS Shield, an Application Load Balancer, and OAuth validation. The server connects to Smartsheet's domain services APIs for transactional operations and queries an Intelligence Layer built on Amazon Neptune and Databricks for cross-project insights. Change events stream through Amazon Kinesis Data Streams and Apache Flink into Amazon S3, following a medallion architecture. Smartsheet deliberately runs Smart Assist (its in-product AI experience) and externally connected AI clients on the same infrastructure, with the same tools and optimizations—the company builds once and every agentic client benefits immediately.

AI traffic exhibits distinctive patterns that drove Smartsheet's deployment and scaling strategy. Agents autonomously orchestrate sequences of tool calls, firing several requests in a second as they work through a task, then going quiet while the model reasons. This bursty traffic demanded a scaling strategy responsive to sudden spikes and sustained throughput. Smartsheet uses ECS Auto Scaling with target-tracking policies that combine traffic volume and compute utilization; each request involves server-side LLM-optimized serialization, not just proxying. Extensive load testing validated that the infrastructure absorbs agent bursts without degradation. On the deployment side, container images are stored in Amazon Elastic Container Registry (ECR) and rolled out via CI/CD through ECS deployment circuit breakers that detect failing containers during rollout and automatically revert to the last stable version, avoiding manual intervention and customer impact. Deployments roll to the smallest region first following AWS Well-Architected principles; after each region, automated end-to-end tests validate tool behavior against the live environment, with canary tests running every 15 minutes to execute a multi-step MCP workflow through the full authentication and gateway path.

Enterprise governance is the gating factor for AI adoption. Smartsheet built governance into the tool framework itself: access control, error handling, and audit trails ship with every tool by default. Access is tiered per organization—administrators can turn on AI access globally, restrict to non-destructive operations only, or open up full write and destructive capabilities. Tools carry MCP protocol annotations like readOnlyHint and destructiveHint so AI clients apply appropriate confirmation flows automatically. The server emits OpenTelemetry signals (logs, traces, and metrics) across the full request lifecycle; every tool invocation captures user, organization, tool name, outcome, and more within privacy constraints. Smartsheet is extending observability with agent-first identity and tracing to correlate context across tool chains. Logs stream through Amazon Kinesis into Amazon OpenSearch Service; infrastructure metrics surface through Amazon CloudWatch; Datadog provides per-tool application performance monitoring (APM) visibility; and PagerDuty handles incident routing. Every invocation also emits a structured analytics event through Amazon Simple Queue Service (SQS) into the Intelligence Layer, closing the feedback loop so production usage data informs tool prioritization.

Security mirrors production APIs: AWS WAF and AWS Shield at the edge, private subnets in a VPC, mutual TLS (mTLS) for service-to-service calls, and an OAuth2 proxy that rejects unauthenticated requests. The API gateway handles authentication and scope validation; domain services handle fine-grained permissions. If a user cannot access a sheet through the UI, they cannot access it through MCP. AI traffic adds a distinctive rate-limiting challenge: a single user question can trigger several tool calls in seconds, and many enterprise users sit behind shared corporate proxies making IP-based rate limiting unreliable. Smartsheet implemented layered rate limiting through AWS WAF—blanket protection at the outer edge, per-user metering using custom aggregation keys on an identity header, and path-specific controls for expensive operations.

Testing non-deterministic AI workflows required new approaches. A conventional API response renders deterministically by the UI; an MCP tool response passes through an LLM first, which interprets it, reasons over it, and generates what the user actually sees. That layer of non-determinism changes what "correct" means for testing. Smartsheet invests heavily in end-to-end workflow tests that include the LLM in the loop, simulating realistic business scenarios—creating workspaces, writing data, querying results, and verifying that the model's interpretation makes sense to the user. These tests run in the CI/CD pipeline (GitLab CI with runners on AWS) and continuously as canary tests against each production AWS Region.

Token consumption is a real cost driver as enterprises scale AI agent deployments. Smartsheet optimizes at three levels. Progressive disclosure caps token consumption per response by dynamically calculating how many rows fit based on column count and data density; a sheet with five columns returns more rows than one with 15 columns, but total tokens stay within budget. Metadata fields (is_sampled, rows_in_sheet, rows_actual, filters_applied) tell the model whether data was truncated and what it is seeing, so the model decides whether to narrow its query with filters. Strongly typed tool schemas, generated from Pydantic models, help prevent hallucinated parameters and wasted calls. A proprietary serialization format reduces token count by 35–47 percent on data-heavy responses. Since launch, Smartsheet has saved over 3 billion tokens using these optimizations.

Smartsheet's MCP server addresses a critical gap in enterprise AI adoption: most internal systems were not built with structured machine-readable interfaces that AI agents can safely and efficiently consume. By layering an AI-optimized interface on top of Smartsheet's existing APIs and central intelligence layer, Smartsheet enables agents to orchestrate complex workflows autonomously—picking up tasks, attaching test results, drafting documentation—all within the same sheets that human users access. This parity between internal Smart Assist and external AI clients like Amazon Quick and Claude Desktop means Smartsheet builds once and every agentic client benefits immediately, avoiding fragmentation.

The infrastructure choices reflect the unique demands of agentic traffic. Traditional API services handle request-response cycles; agentic traffic is bursty, with agents firing multiple tool calls in seconds as they reason through tasks. Smartsheet addressed this by running the MCP server on AWS Fargate with auto-scaling policies that combine traffic volume and compute utilization, not just proxy throughput. The deployment strategy—rolling to the smallest region first, validated by canary tests every 15 minutes—protects active agent sessions from disruption. The observability layer extends beyond traditional monitoring to capture context across tool chains, since a single user request can produce a chain of tool calls and failures often trace back several steps.

Token cost is the defining economic constraint of AI agent deployments. Smartsheet's three-level optimization strategy—progressive disclosure that caps response size, strongly typed schemas that prevent hallucinated parameters, and proprietary serialization that cuts token count by 35–47 percent—directly attack this problem. The 3 billion tokens saved since launch demonstrates that the server design prioritizes not just correctness but efficiency at scale.

No comments yet. Be the first to share your thoughts!