Amazon Bedrock Knowledge Bases hits 90.5% claim retrieval recall

Amazon's post walks through building a conversational claims assistant using Amazon Bedrock Knowledge Bases, a managed retrieval-augmented generation (RAG) service that takes claim documents stored in Amazon S3 and makes them searchable in plain language. The setup handles PDFs, Word files, and text notes, and pairs each document with a metadata sidecar file so the assistant can filter by claim ID, type, status, amount, and date filed. This addresses a real operational mess: claim answers live in adjuster diaries, repair estimates, police reports, and payment ledgers rather than a single database field, and revised estimates or reversed payments can contradict earlier records.

The retrieval side relies on AgenticRetrieveStream, which lets the model plan an answer, split multi-part questions into sub-queries, run several retrieval rounds, and check whether the evidence is sufficient before generating a response. It streams trace events that expose the retrieval plan and attaches citations linking each answer segment to a source document, so contact center agents can verify a source before repeating it to a policyholder and supervisors can audit how an answer was reached.

The results Amazon reports come from a 30-document synthetic corpus and 40 test questions, with automated grading rather than human review. The post itself warns these are directional findings and that human review on a real corpus is needed before exposing an assistant to policyholders. The real test for insurers weighing this approach will be whether retrieval and citation quality hold up on their own claim files and whether the metadata filters correctly enforce access boundaries derived from authenticated sessions.