AI-written citations 38.6% fabricated, study finds

This benchmark uniquely combines machine resolution with hand verification, correcting its own tool's errors before publishing. The authors deliberately avoid ranking models, citing wide intervals from only two reports per model. The finding that fabrication doubles on thinner topics suggests a mechanism: models excel at citation format, not content. For readers, the practical takeaway is to verify references mechanically, as plausibility is not a reliable filter. The study's limits — nineteen reports, two topics, no browsing — mean the 38.6% figure is specific, not universal. However, the consistency across all models underscores the issue's breadth. The authors also note that retrieval-backed tools, which browse while writing, would likely produce different results, hinting at a future benchmark. This work positions existence as the floor of citation quality, not the ceiling, as resolving sources can still be misattributed or weak.