FinTech builder deploys 9-model AI council to prevent hallucinations in daily market digest

MarketDaily is a personalized daily financial email digest launched 2026-05-19 that allows subscribers to select their US and Taiwan stock holdings, then generates and sends them a personalized HTML report twice daily at fixed times. As of the article's writing, it has 21 subscribers and 75 days of production history with 1,800+ commits. The failure tolerance is near-zero: financial content landing in inboxes before market open cannot hallucinate price targets or meaningful content, or user trust is permanently broken.

The builder's core architecture places deterministic code at the center. Market data flows into a "structure prior" layer where price direction relative to MA20 and MA50 moving averages is decided entirely by code, not by any LLM. An LLM seat that recommends a direction contradicting this structure is demoted to neutral by a plain if statement. This boundary confines the council to where models are actually useful: thesis, counter-risk, and conviction within the allowed structural range.

The council itself consists of 9 configured free-tier seats across 7 providers: Gemini (2 seats), Groq, Ollama (a local 14B model on the author's GPU), Cloudflare Workers AI (2 seats), OpenRouter, Cerebras, and OpenAI. Seats with no API key auto-skip. On a typical day, about 5 independent voices live in the council. Each seat returns JSON output with four fields: lean, conviction, thesis, and key_risk. Quorum requires at least 2 seats responding, or there is no council verdict and the stock falls back to a single-model path. The council is designed around vendor independence: Groq's free tier is limited to 8,000 tokens per minute and 200,000 tokens per day, so the council seat uses a different model than the report-generation chain to avoid exhausting one shared quota bucket (a earlier incident burned 197,364 of 200,000 daily tokens, leaving only 45 report calls and forcing a 144-second-per-call backup, making one evening's digest 1 hour 35 minutes late). The local GPU seat has zero quota and zero network dependency, ensuring survival if cloud DNS blips or every vendor quota dies at once. One model that was discovered to rewrite prices (e.g., 385.25 became 385.00) is permanently banned from anything touching prices but remains allowed in the council, because council output is JSON opinions with no numbers in them—blast-radius design beats model trust. Seats have circuit breakers: if a seat hits quota exhaustion, a missing API key, or 3 consecutive failures, it disables for the round. HTTP 402 (billing wall) kills a seat on the first strike; retrying a payment error is pure waste.

The judge layer synthesizes all seat opinions and is instructed to synthesize consensus and disagreement rather than parrot the loudest seat. The judge has its own fallback chain: free-tier Gemini lite → Groq → if both die, take the highest-conviction seat's thesis verbatim. The entire council is fail-safe; any failure returns partial results and never blocks the send. The system enforces two different guarantees: "never miss a send" (handled by fallback architecture) and "never send garbage" (handled by the audit layer).

Before send, a function called audit_digest() runs 31 named deterministic checks against the final HTML. Every check maps to a real way a user was angry or would have been angry. Checks include tense discipline (at 7am you cannot write "Taiwan stocks rose today" because the market opens at 9), holdings coverage (every stock the user selected must have an action card), fabrication detection (placeholder XXX tokens, fake URLs, earnings estimates not in the data feed), prompt-instruction leakage, truncation detection, and undefined CSS classes. HIGH failures trigger a 60-second wait (because free tiers are per-minute limited and a 5-second retry just hits the same 429 window), then regeneration with a stronger model forced to the front. If it still fails, the system ships a deterministic fallback—a plain code-assembled report deliberately containing no price levels.

Three production incidents shaped the system. First, the 429 incident: Gemini's daily quota ran out, and the retry-with-backoff logic burned ~100 seconds per call across the whole run, sending the digest 5 hours late. The fix: two consecutive 429s on one model means quota is dead for the day, and the entire model is fused for the run. Retry logic correct for transient errors is actively harmful for quota errors; you must classify.

Second, the CSS incident in two acts. LLMs asked to "reuse the existing CSS classes" would freestyle near-miss names like "news-title" instead of "news-headline," leaving no stylesheet rule and rendering unstyled sections. The builder added an undefined_css_class HIGH check. One week later, a Monday-edition prompt lacked a verbatim skeleton, so every model invented classes, all 12 users hit the check, the retry hit it too (same prompt, same disease), and 9 users got the degraded fallback version. The defense became the incident. The real fix was a deterministic repair layer before the check: map known near-miss names back to real classes, strip unknown ones. This taught two lessons: telling an LLM "use the existing classes" is not a spec—only a verbatim skeleton is; and before adding any HIGH check, ask what happens if everyone fails it at once.

Third, the depth-collapse incident. All free quotas fused one morning, generation fell to the weakest models, and per-stock reasoning collapsed from 129–165 characters to 48–64. Format was perfect, prices were present, every check passed. A human caught it. The fix was statistical: calibrate against real production history (normal days had median reason length 107–197 chars; the bad day was 48), and flag any run whose median drops below 80 as systemic collapse. Checking format is easy; checking depth requires a baseline from your own good days. A second horizontal defense: if the same HIGH check fires for 3 users in a single run, page the admin immediately. Per-user retry→fallback chains handle individual failures fine and systemic failures terribly—they degrade everyone quietly, one user at a time.

After 75 days, 34 days of which included the council (launched day 42, 2026-06-30), and two sends a day, the system has achieved the goal: twice-daily financial reports with no hallucinated price targets leaving the system. The author notes a correction dated 2026-08-02: the article states "31 checks," but the real number is 30. One "check" exists only in a docstring usage example and can never fire; the author's own verification grep counted it anyway—exactly the kind of failure the next post will dissect. A post announcing the full taxonomy of the 30 checks and their failure modes is planned.