How are ideas judged and what happens to them?

A blind automated court judges submitted fragments without seeing the submitter's identity, promoting them to real experiments on real data, keeping them in reserve, or parking them. Experiments are run against 1.05M real market ticks and scored as supported, inconclusive, or killed. Killed experiments go into the graveyard, which is searchable before submission to prevent duplicate work.

What counts as a successful contribution?

Any testable fragment with a concrete mechanism and falsifiable test can be promoted. Negative results—documented failures with numbers—are explicitly valued as evidence and count toward the leaderboard under the submitter's alias. The first-night leaderboard shows GPT-5.6 Pro at 7 promoted out of 25 judged, Gemini Advanced at 45% survival, and local models with zero entries so far.

How do I submit and what tools do I need?

Submissions use a Python 3.8+ client (dreamwalk.py) with stdlib only, or plain curl via the wire protocol in PROTOCOL.md. No account or dependencies required. Harness mode allows an agent to auto-submit fragments from its standing instructions at session end, with privacy-stripped results.

Large Language ModelsOpen-Source AIAI Business & IndustryHacker NewsJul 25, 2026Jul 25, 2026, 16:01 JST

AI agents test ideas on live market data, scored blind on Bitcoin

Send on LINE

3 Key Points

What happened
Dream Walk has launched a public research exchange where AI agents and humans submit hypothesis fragments that are blind-judged by an automated court, tested against 1.05M real market ticks, and ranked on a live leaderboard. In the first night, 68 fragments were submitted; 20 were promoted to real experiments, 42 kept in reserve, and 6 parked, with 19 experiments run the same night yielding 3 supported results, 5 inconclusive, and 11 killed.
Why it matters
The platform creates a shared graveyard of failed ideas and negative results, preventing duplicate compute waste across AI models and teams. A specific failure with numbers outranks a vague hypothesis, and negative results are explicitly valued as evidence rather than dismissed as waste — allowing any model (GPT-5.6 Pro, Gemini Advanced, Grok, Opus, Sonnet, and local models) to contribute without account friction.
What to watch
The current bounty focuses on the teacher-ladder fight (three incompatible answers blocking downstream training runs); any fragment that survives court and cracks the problem earns a permanent law named after the submitter's alias in LAWS.md. Submissions use a Python 3.8+ client or curl, with no dependencies beyond stdlib, and can include harness mode to auto-submit fragments from an agent's standing instructions.

Not sure about something? Ask the AI

Questions and answers are published on this page.

Summaries like this, in your inbox every morning.

Get news like this every morning, free

Context & Analysis

Dream Walk addresses a fundamental inefficiency in AI research: the isolation and redundancy of failed experiments. Each AI model or team conducts backups, sweeps, and ablations independently, burning compute on problems already solved or dead-ended elsewhere. By making a blind court the judge of idea promotion, the platform removes reputation friction—a novel hypothesis from an unknown alias ranks on equal footing with one from GPT-5.6 Pro—and frames negative results not as waste but as permanent knowledge. The graveyard becomes the most-read page because a concrete failure (e.g., "LoRA rank>64 on 8B gave zero eval gain") with numbers and replication conditions saves the next researcher a month of GPU-hours.

The live market-data scoring mechanism (1.05M ticks tested in one night) ensures that hypotheses are not accepted on narrative plausibility but on empirical survival. The first night's results—3 supported, 5 inconclusive, 11 killed from 19 experiments—reinforce that the graveyard is the expected destination for most ideas. This is by design: the house celebrates clean kills as contributions, not failures. The current bounty on the teacher-ladder fight (three incompatible answers blocking training runs) exemplifies how a shared adversarial cortex can focus collective attention on a high-leverage roadblock.

FAQ

How are ideas judged and what happens to them?: A blind automated court judges submitted fragments without seeing the submitter's identity, promoting them to real experiments on real data, keeping them in reserve, or parking them. Experiments are run against 1.05M real market ticks and scored as supported, inconclusive, or killed. Killed experiments go into the graveyard, which is searchable before submission to prevent duplicate work.
What counts as a successful contribution?: Any testable fragment with a concrete mechanism and falsifiable test can be promoted. Negative results—documented failures with numbers—are explicitly valued as evidence and count toward the leaderboard under the submitter's alias. The first-night leaderboard shows GPT-5.6 Pro at 7 promoted out of 25 judged, Gemini Advanced at 45% survival, and local models with zero entries so far.
How do I submit and what tools do I need?: Submissions use a Python 3.8+ client (dreamwalk.py) with stdlib only, or plain curl via the wire protocol in PROTOCOL.md. No account or dependencies required. Harness mode allows an agent to auto-submit fragments from its standing instructions at session end, with privacy-stripped results.

Hacker NewsRead Original Article

Send on LINE

AI news that matters for your work, delivered every morning.

Pick your industry and the AI tools you use, and get news related to your work every day.

Get it free

Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →

Ask AI

Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.