AIToday
Large Language ModelsAI Coding AssistantsZenn AI/MLPublished: Oct 6, 2026, 10:01 JST

15 FAQs, no RAG: Claude Haiku 4.5 bot for tax office

15 FAQs, no RAG: Claude Haiku 4.5 bot for tax office

3 Key Points

  1. What happened

    A developer built an inquiry-desk bot for a fictional tax accountant's office on Next.js 16 and the Anthropic API, embedding all 15 FAQs straight into the Claude Haiku 4.5 system prompt with no vector search or database.

  2. Why it matters

    The developer judged that embedding all 15 FAQs in the prompt each time kept both accuracy and cost sufficient, making the added setup of a retrieval setup unnecessary at this scale.

  3. What to watch

    The approach hinges on the developer's stated boundary of roughly dozens to about a hundred FAQs before token costs make the current design impractical. Watch whether the planned move to a database and embedding-based retrieval follows once that threshold is crossed.

WHO IT HITSThis is most relevant to developers and small professional-services firms weighing whether to build retrieval infrastructure for a limited FAQ set; the author's experience suggests that below a few dozen FAQs, a direct prompt-embedding approach may be adequate, with a switch to a more complex setup deferred until volume grows.

Not sure about something? Ask the AI

Questions and answers are published on this page.

Summaries like this, in your inbox every morning.

Context & Analysis

The project was intended as a small, working proof of concept for selling generative-AI adoption support to professional-services firms, a field the developer says is hard to win trust in without real examples. The tax-accounting domain was picked because almost every office has a front-line inquiry desk, and the constraint of never giving a wrong answer is easy to grasp. The system prompt steers the model to rephrase only what is in the FAQs; anything outside them, such as individual tax calculations or specific numerical simulations, is treated as a case for a human, with the bot directing the user to a contact form. The developer deliberately kept the scope narrow to two choices — answer from the FAQ or hand off — and reports no significant stumbling blocks, attributing that to the simple architecture and limited scope rather than to any inherent ease. The stated plan is to decouple FAQs from code into a database, add an admin interface for office staff, and move toward embedding-based retrieval as volume grows, with the trigger being whether all FAQs can still be passed in every time without practical trouble.

FAQ
What technology stack was used for the bot?
The bot was built with Next.js 16 (App Router), TypeScript, Tailwind CSS v4, and the Anthropic API (Claude Haiku 4.5).
Why was Claude Haiku 4.5 chosen over a more capable model?
The developer said FAQ answering involves no complex reasoning — mostly matching the user's question to an embedded FAQ and phrasing a natural reply — so response speed and cost were prioritized over more advanced model capabilities.
What are the limitations of this approach?
Every FAQ addition or edit requires a code change and redeployment, so non-engineer office staff cannot update it themselves. Token costs become non-negligible at hundreds of FAQs, and serving separate FAQ sets per office would not fit the current one-office-per-codebase structure.

AI news that matters for your work, delivered every morning.

Pick your industry and the AI tools you use, and get news related to your work every day.

Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →

Ask AI

Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.

Questions and answers are published on this page.

Related Articles

Next articleAleph Alpha releases Kolibri, a sovereign open-weight model