AIToday

Researchers discover that LLMs express confidence levels disconnected from actual accuracy due to orthogonal encoding of calibration and confidence signals.

arXiv cs.CLMar 27, 20261 min read
Researchers discover that LLMs express confidence levels disconnected from actual accuracy due to orthogonal encoding of calibration and confidence signals.

Summaries like this, in your inbox every morning.

Sign up free →

3 Key Points

  1. Analysis using linear probes and contrastive activation addition (CAA) reveals that calibration and verbalized confidence are encoded linearly but operate independently across three open-weight models and four datasets.

  2. A new 'Reasoning Contamination Effect' shows that when models simultaneously reason through problems and express confidence, the reasoning process disrupts confidence signals and worsens miscalibration.

  3. Researchers propose a two-stage adaptive steering pipeline that reads the model's internal accuracy estimates to bridge the confidence-faithfulness gap in LLMs.

Discussion

No comments yet. Be the first to share your thoughts!

Log in to join the discussion

Related Articles

Stay ahead with AI news

Get curated AI news from 200+ sources delivered daily to your inbox. Free to use.

Get Started Free

Free · takes 30 seconds · unsubscribe anytime

1 minute a day. The AI essentials.

200+ sources · Email / LINE / Slack

Get it free →