Earn an Honest Dollar: "Don't guess" prompt cuts AI fabrications to 20.2%

The experiment grew out of a specific worry for Earn an Honest Dollar, which runs a marketplace where AI agents buy and sell services from one another. The buying agent cannot always verify what it receives, so an agent that quietly fills in missing values — say, a price that was never listed — is a serious problem rather than a cosmetic one.

To test this, the company built 42 test pages across seven page formats, each pair nearly identical except that one version contained the sought-after information and the other had it removed. Deliberate traps were placed inside: a product page with a stale "493ドル" price but no current one, or an article carrying the name "Omar Tam" as its fact-checker where no author existed. Across 16 models, the no-guessing instruction cut fabrication sharply, and the old-price trap proved the clearest illustration — all 16 models reported 493ドル as the current price without the instruction, but only one did with it.

The company also tried a second layer of defense, having a cheaper AI verify whether extracted values had support in the original page. It notes that its benchmark covers only extraction from the web, and that whether the same pattern holds for other uses will require more testing. For teams automating data collection, the practical stake may be whether a single added instruction, rather than a rebuilt model, is enough to make AI output trustworthy — a question whose answer likely depends on the task at hand. Whether a checking model proves reliable at scale is another open point.