1830 railroad game trains AI to ace Finance-Agent benchmark

Good Start Labs grew out of a 2025 Twitch stream where Alex Duffy watched frontier models play Diplomacy and noticed each model behaved differently: OpenAI's o3 won by planning a future betrayal, while Claude's Opus 4 refused to lie and got destroyed. That observation led Duffy to the idea that games with verifiable outcomes could teach AI models skills like strategic thinking. He later wrote on Every that fine-tuning a model on Diplomacy improved its performance on customer support and industrial operations benchmarks. The company was spun out of Every last October with $3.6 million from General Catalyst, Inovia, Every, and angel investors.

The 1830 experiment is the latest step in that line of work. The game, a nineteenth-century railroad strategy title, includes a stock market mechanic and mirrors a finance workflow: models find information in a database, put it into an Excel file, reason over it, create functions, and calculate an answer. The published results compared single-turn question answering with a multi-turn terminal agent that uses tools to explore, plan, and adapt in real time. Both improved in-game, but only the terminal-agent design improved performance on the Finance-Agent benchmark. Meanwhile, Duffy says newer models still diverge on personality axes like betrayal and collaboration, and that for treating the environment as curriculum the harness matters more, not less.

The stake for Good Start Labs is whether it can sell more than custom data and environments — its main customers are frontier labs buying reinforcement learning data. The company is also building a general model from its game-specific expert models, aiming for what Duffy calls a general game intelligence. Whether that broader real-world transfer materializes is the open question Duffy himself acknowledges, and it is likely to determine how far this approach reaches beyond structurally similar tasks.