AWS benchmarks Amazon Bedrock Knowledge Bases vector stores

AWS's comparison places three supported customer-managed backends side by side: Amazon OpenSearch Service, Amazon Aurora PostgreSQL with pgvector, and Amazon S3 Vectors. The product catalog search test used the Shopping Queries Data Set (ESCI), indexing 1,215,851 unique US products and sampling 5,000 queries. A separate test used over 6 million English Wikipedia articles, chunked into 300-token segments, to measure Amazon S3 Vectors across index sizes from 5,314 to 1,000,413 vectors.

The product search results show a clear trade-off between index size and retrieval quality. While 512-float matched the 1024-float baseline, dropping to 256 dimensions showed a measurable 4.4 percent quality loss. Binarization cut the index by 13.4× at 1024 dimensions but cost 5.2 percent NDCG, and that penalty grew to 28.3 percent at 256 dimensions. Disk mode reduced the index by the same 13.4× as 1024-binary while retaining more quality, but at roughly 3× higher latency (99 ms vs 31 ms p50). Enabling hybrid search provided a consistent +6.0 percent NDCG lift over semantic-only search, which was enough to let 1024-binary with hybrid (0.3658) edge past 1024-float with semantic-only (0.3633) at 13× less memory.

For teams deciding, the practical question centers on which trade-off their workload can tolerate. Ecommerce platforms needing low latency and faceted navigation are pointed toward Amazon OpenSearch Serverless, while deep research agents that run for minutes or hours are pointed toward Amazon S3 Vectors, whose latency at 1 million vectors stayed under 510ms at p99. The benchmark notes that the absolute hybrid lift may differ on natural-language questions, since this dataset favors keyword matching and the 0.7/0.3 fusion weight was set for demonstration rather than tuned.