AWS SageMaker skips S3 for small inference requests

What happened

Amazon SageMaker AI Async Inference now accepts a new Body parameter that allows inference payloads to be sent inline in the API request itself, removing the requirement to upload input data to Amazon S3 before each invocation. The feature supports payloads up to 128,000 bytes and is available in 31 commercial AWS Regions.

Why it matters

Previously, every async inference request required two steps—uploading to S3 and then invoking the endpoint—which added latency, complexity, and cost even for small payloads. The inline approach removes one network round-trip, eliminates the need to provision an input S3 bucket, and avoids per-request S3 PUT charges. It also simplifies error handling by returning validation feedback synchronously in a single API call.

What to watch

The feature works only for payloads up to 128,000 bytes; larger payloads (images, audio, multi-MB documents) still require the S3 InputLocation approach. Customers can adopt the new Body parameter immediately by updating to the latest Boto3 SDK, with no changes needed to existing async endpoints.