What does the hidden prompt method actually do?

Conference organizers embed concealed instructions into papers sent to peer reviewers that tell large language models to use telltale phrases—such as "This work addresses the central challenge" and "The claims of the paper"—in their reports, making AI-generated reviews identifiable.

How effective has this approach been?

At ICML 2026, a similar hidden-prompt effort identified hundreds of reviewers misusing AI, leading to rejection of their reviews. The conference desk-rejected just under 500 papers over LLM review policy violations, about 2 percent of total submissions, and researchers expressed overwhelming support for the strategy.

What is the policy on AI use in peer review at these conferences?

NeurIPS bans peer reviewers from uploading papers they referee to AI chatbots because it breaches confidentiality, though reviewers are permitted to use AI chatbots for background research. Many other publishers similarly ban AI use in peer review.

Back to articles

NeurIPS uses hidden AI prompts to catch cheating reviewers

Hacker News20h ago5 min read

Key takeaway

Academic conferences NeurIPS and ICML 2026 are inserting hidden instructions into peer-review papers to catch reviewers who use AI chatbots to write reviews, violating confidentiality and peer-review norms. One similar effort at ICML caught hundreds of offenders and earned broad researcher support, but critics warn the tactic presumes bad faith and may backfire by punishing honest reviewers who mistake organizer prompts for author misconduct.

Summaries like this, in your inbox every morning.

3 Key Points

What happened
The 40th Annual Conference on Neural Information Processing Systems (NeurIPS), scheduled for Sydney in December 2026, has embedded concealed instructions for large language models in peer-review papers to detect reviewers uploading those papers to AI chatbots in violation of policy. The hidden prompts instruct LLMs to use specific phrases like "This work addresses the central challenge" in their reports, making AI-generated reviews identifiable.
Why it matters
Peer review depends on human judgment and confidentiality; uploading papers to AI chatbots breaches both. A similar effort at ICML 2026 in Seoul caught hundreds of reviewers misusing AI and led to rejection of reviews, suggesting the method can enforce standards that many publishers now require. However, some researchers argue the approach—treating reviewers as suspects rather than building trust—risks corroding the culture peer review needs.
What to watch
At ICML 2026, hidden prompts led to desk rejection of just under 500 papers over LLM review policy violations, representing about 2 percent of total submissions. Some reviewers have already flagged the NeurIPS prompts after spotting them when converting PDFs to Word, raising the possibility that papers may be incorrectly rejected if reviewers don't realize the prompts came from organizers, not authors.

FAQ

What does the hidden prompt method actually do?: Conference organizers embed concealed instructions into papers sent to peer reviewers that tell large language models to use telltale phrases—such as "This work addresses the central challenge" and "The claims of the paper"—in their reports, making AI-generated reviews identifiable.
How effective has this approach been?: At ICML 2026, a similar hidden-prompt effort identified hundreds of reviewers misusing AI, leading to rejection of their reviews. The conference desk-rejected just under 500 papers over LLM review policy violations, about 2 percent of total submissions, and researchers expressed overwhelming support for the strategy.
What is the policy on AI use in peer review at these conferences?: NeurIPS bans peer reviewers from uploading papers they referee to AI chatbots because it breaches confidentiality, though reviewers are permitted to use AI chatbots for background research. Many other publishers similarly ban AI use in peer review.

Discussion

No discussion yet for this article

Stay ahead with AI news

Get curated AI news from 200+ sources delivered daily to your inbox. Free to use.

Get Started Free

Free · takes 30 seconds · unsubscribe anytime

1 minute a day. The AI essentials.

200+ sources · Email / LINE / Slack

Get it free →