Orca Security's Discovery AI cuts query costs 90% on Amazon Bedrock

Orca Security's Sonar query system covers over 900 cloud models, and querying it required exact model names, parent-child hierarchy navigation and selection from 18 operators. An initial proof of concept loaded the full schema into a single prompt on the largest model, which produced strong output quality but consumed roughly 2 million input tokens per workload and ran at around 60 seconds per response.

The production Discovery AI pipeline replaced that with a nine-stage architecture on Amazon Bedrock, combining retrieval-augmented generation with schema-aware query construction. A Quick Matcher handles structured inputs like IP addresses and CVE IDs without model inference, while a Model Selector on Anthropic Claude Haiku 4.5 narrows the schema down to the models a request actually needs. Example retrieval proved the biggest accuracy lever: top-20 nearest-neighbor search using Amazon Titan embeddings, reranked to five examples with Cohere Rerank 3.5, gave the Generator a small, diverse set without trading off coverage.

The team also built a continuous evaluation harness and a domain-expert labeling system, so prompt, model or retrieval changes pass automated checks before reaching production. Or Gabay, Director of AI at Orca Security, described the labeling UI as the single most important investment for long-term quality, turning Discovery AI into a living system rather than a static deployment.