AIToday
Large Language ModelsHacker NewsPublished: May 5, 2026, 04:00 JST1 min read

snc-core reduces hallucination rate by 52% on HumanEval with Qwen2.5-Coder-7B via inference-time governance layer

snc-core reduces hallucination rate by 52% on HumanEval with Qwen2.5-Coder-7B via inference-time governance layer

3 Key Points

  1. snc-core wraps any decoder-only LLM with a governance layer that reduces hallucination rate from 16.5% to 7.8% on HumanEval with Qwen2.5-Coder-7B (z = 2.12, p < 0.05), requiring no model retraining.

  2. The layer combines three signals: confidence elicitation (instructing the model to emit a self-confidence score), behavioral clustering (sampling K = 5 candidates at temperature 0.8 and grouping by output equivalence), and trust thermodynamics (a closed-form score that discounts as candidates diverge, compared against a user-supplied threshold θ).

  3. At conservative threshold θ = 0.65, the system achieves 92.17% precision while recovering five previously failing test cases, though nine residual failures stem from adversarial mode collapse where multiple candidates make the same systematic error.

  4. Available via pip install snc-core for Python 3.9+ with no mandatory dependencies beyond the standard library; supports Ollama-served local models and OpenAI-compatible APIs.

Ask the AI about this article →

Get the latest Large Language Models news every morning

For example, today's edition would include:

  • Winamp Group's Jamendo expands AI music lawsuits, adds six more targetsYahoo Finance AI · 1h ago
  • Anthropic resets Claude usage limits with Fable 5.1 launchITmedia AI+ · 5h ago
  • Salesforce and Anthropic unveil Claudeforce, integrating CRM into ClaudePublickey · 5h ago

AI-summarized, only the topics you pick — one digest a day via Email, Slack, or Discord.

Free · takes 30 seconds · unsubscribe anytimeWhat is AIToday? →

Ask AI

Ask AI anything about this article. Q&As are published on this page for other readers too.

Related Articles

Next articleElon Musk and Sam Altman face off in court over OpenAI's conversion to for-profit structure