Amazon Bedrock launches prompt optimizer for multiple models at once

Amazon Bedrock has introduced Advanced Prompt Optimization, a tool designed to address one of the most time-consuming aspects of building generative AI applications: migrating prompts to new models or optimizing them for current ones. The problem is acute: when a new, faster, cheaper, or more capable model becomes available, teams must decide whether to migrate—a decision that is straightforward in theory but rarely in practice. Customers currently spend days to weeks optimizing prompts and re-evaluating responses during migration, going through the same cycle repeatedly: rewrite the prompt, run it against test cases, compare results, tweak, and repeat. This manual work multiplies across every prompt template in production and every model candidate worth evaluating, creating a bottleneck that scales with business ambition.

The consequences are severe. Model lock-in develops when teams avoid migration because the re-tuning effort is too high, even when newer models would reduce latency and inference cost. Underperformance occurs when prompts tuned for one model don't unlock the full capability of another. Regression blindness—the inability to distinguish "different" from "worse"—means teams ship degraded outputs unknowingly. And slow iteration cycles keep engineers stuck in evaluation loops instead of building features, as every prompt change requires manual A/B testing.

Advanced Prompt Optimization works by operating a reinforcement learning-style feedback loop without changing model weights. You provide a prompt template, example user inputs (text or multimodal), optional ground-truth answers, and an evaluation metric. The optimizer sends your template and inputs to your chosen inference models, evaluates responses against your metric, then rewrites the prompt. This loop repeats—evaluate, rewrite, evaluate—iteratively optimizing the prompt against your metric. The output includes the original and optimized prompts, evaluation scores, time-to-first-token (TTFT) per sample, and inference cost estimates at on-demand pricing. A single job can target up to 5 models at once, giving you an optimized prompt for each candidate and a direct comparison of quality, latency, and cost.

The tool supports three evaluation modes, each serving different use cases. AWS Lambda functions are best for concrete metrics like accuracy, F1, ROUGE, or JSON match, requiring you to provide a function with compute_score logic. LLM-as-a-Judge suits open-ended tasks like summarization, reasoning, and generation, where you provide a rubric prompt and select a judge model. Steering criteria work for brand voice, format, and safety constraints, letting you specify up to 5 natural language criteria. If you omit all evaluation fields, the system applies a built-in default combining accuracy, answer completeness, and writing style scoring, though you can create composite metrics with weights for multiple objectives. Advanced Prompt Optimization also supports multimodal inputs—PNG, JPG, JPEG, GIF, WebP, and PDF files referenced through Amazon S3 URIs—enabling optimization for document analysis, image classification, visual question answering, and other tasks combining text with visual input.

Users can interact with the tool via the Amazon Bedrock console, API, or boto3 SDK. The console workflow involves five steps: creating a job, selecting inference models (baseline plus up to 4 candidates), preparing a JSONL input dataset with prompt templates and evaluation samples, uploading the file or importing from S3, and submitting the optimization. Results are displayed in a summary view showing headline before-and-after scores, and a details view showing the original and optimized prompts side by side, per-sample metrics, and full token and latency breakdowns. Example runs demonstrated substantial improvements: a NESTFUL function-calling task on Amazon Nova 2 Lite increased accuracy from 0.267 to 0.647, while an XSum summarization task on Claude Haiku 4.5 improved quality score from 0.550 to 0.743 while output tokens dropped from 51 to 28.