Orca Security cuts natural-language query costs 90% using a staged Bedrock agent pipeline
In short: Orca Security built Discovery AI, an Amazon Bedrock-based agent that turns plain-English security questions into structured queries against its Sonar system, which tracks over 900 cloud resource types. Rather than relying on one large model with the full schema stuffed into every prompt, the production version splits the work across nine specialized stages and multiple foundation models. The result: dramatically lower cost, faster responses, and better accuracy on complex queries, now running live since March 2026.
This summary was generated automatically by AI from AWS Machine Learning's publication. It is our own text, not a copy of the original — facts, figures and quotes belong to the source, linked above and below.
What changed?
- 1Pipeline stages: pattern-based fast matching, safety guardrail, embedding retrieval, reranking, model-scope selection, agentic filter generation, validation with retries, response output
- 2Claude Haiku 4.5 drives model selection and filter generation; Nova 2 Lite runs the guardrail and validator; Titan generates embeddings; Cohere Rerank 3.5 reorders retrieved examples
- 3Per-workload input tokens dropped from roughly 2 million to 140,000, a 93% cut
- 4Total query cost down 90% versus the original single-prompt prototype
- 5Average response time fell from about 60 seconds to 15 seconds, a 75% reduction
- 6A human-labeling workflow lets experts correct outputs and feed them back into the example set without a new deployment
- 7Live in production since March 2026, processing thousands of queries daily
Why it matters
It shows that splitting an LLM agent into specialized, right-sized stages — rather than one giant prompt — is what actually makes natural-language-to-structured-query systems cheap and fast enough for real production traffic at scale.
What it means for AI agents and contact centers
The same pattern applies directly to voice and chat agents that need to query large schemas (CRM records, call logs, ticketing systems): use a cheap fast-path matcher for simple lookups, route only relevant schema pieces to the generation model, and reserve the strongest model for the hardest reasoning step — this is a concrete way to lower per-call cost and latency.
🧪 Worth Testing
The stage-by-stage model allocation and retrieval-then-rerank approach is directly transferable to building cheaper, lower-latency agents for structured data lookups in contact-center workflows.
Sources
- AWS Machine LearningOfficialPrimary sourceOriginal article →„Natural language security queries at Orca Security using Amazon Bedrock“13 Oct 2026, 19:03
- Published by source
- 13 Oct 2026, 19:03
- Found by our system
- 13 Oct 2026, 19:11
- Summary generated
- 13 Oct 2026, 19:17
This article was written by AI from the original source. Facts, numbers and prices come from the source; missing values are marked “Not specified”. Legal notice, copyright and privacy