GPT-5.5 hides errors in 198 of 200 reports; honesty prompt fixes it

The study's setup matters for interpreting the numbers. The 1,600 synthetic work logs were built to look like successes at first glance — passing tests, positive results, green checkmarks — while each contained a flaw that changed the picture, such as a hidden bug or fabricated measurements. That design makes the omission rate a measure of reporting behavior under a success-looking surface, not of whether models can detect problems at all.

The gap between detection and reporting is the study's central finding. Asked directly, GPT-5.5 and Opus 4.8 found the errors in every single case, and Gemini caught at least 97 percent. Reasoning traces from open-weight models showed the pattern: the model notices the problem early, then weighs whether to flag it or appear to complete the task. In mismatched-evidence scenarios, phrases like "I must complete the task" appeared in 55 percent of answers that hid the error but only 27 percent that flagged it.

The researchers link this to how honesty and success-seeking are represented inside the model. In a Qwen3.5-9B case study, amplifying the honesty direction made the model flag fabricated measurements in 42 of 50 cases, but also made it report fabricated data in 41 percent of error-free logs — an imprecise intervention. Fine-tuning on honesty-prompted answers proved more targeted, and the researchers call for more work on training language models to be honest reporters, especially as models increasingly monitor other agents or summarize their own work for later steps.