
What happened
ArisGlobal reversed the order of its NavaX requests on Amazon Bedrock so the shared prompt prefix is cached first. It now serves 65% of input tokens from cache, cutting effective input cost by about 50% and monthly inference spend by about 40% at more than 19 billion tokens per month.
Why it matters
Caching the repeating shared prefix rather than the larger per-case document lets ArisGlobal reuse the same cache entry across requests, cases, and tenants. That supports onboarding more enterprise tenants without proportional cost growth, the company reports.
WHO IT HITSEnterprise IT and AI platform teams running multi-tenant generative AI workloads on Amazon Bedrock may see similar savings by reordering requests to cache stable, repeated prompt prefixes first. The approach requires putting shared instructions and schemas ahead of per-case documents.
Summaries like this, every morning.
ArisGlobal's NavaX engine handles adverse-event reports for pharmaceutical companies, extracting around 130 fields from multipage PDFs across multiple sequential model requests. Each request carries system instructions, customer-specific rules, an extraction schema, and the source document. Two inputs repeat at different scopes: the shared prompt prefix repeats across requests, cases, and tenants using the same agent, while the source document repeats only for requests tied to one case.
The initial design cached the source document because it was the largest part of a request, leaving the growing shared prompt prefix outside the cache. As ArisGlobal added enterprise tenants with distinct compliance requirements, the shared prefix grew substantially over 6 months, and per-case inference costs rose more than proportionally. Because the prefix was reprocessed on every request, its growth compounded with transaction volume.
Rather than adding another checkpoint, ArisGlobal changed the order: the shared prompt prefix comes first, the checkpoint follows, and the document comes last. This trades limited document reuse within one case for prompt reuse across the fleet. The company reports that 65% of input tokens on a typical request are now served from the shared prompt cache, with cache reads consistently exceeding writes in production monitoring.
Pick your industry and the AI tools you use, and get news related to your work every day.
Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →
Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.
Tricentis's Quality Transformation Report 2026 found 60% of companies globally and 65.6% in Japan deploy untes…

At Microsoft Ignite 2026, Microsoft announced MAI Code 2, which reaches 98% of GPT 6.1 Sol's and 96% of Claude…

Jordan Belfort's AI voice startup Tone has brought in about $1 million in revenue and targets $20 million to $…

Microsoft introduced Copilot Workspaces, a collaborative space in its Frontier early-access program where Word…

Earlier in November, Meta chief AI officer Alexandr Wang shared an AI-generated clip on Instagram promoting Mu…

At a roundtable after his Microsoft Ignite keynote at San Francisco's Chase Center, Satya Nadella said he is "…
