Unbounded Labs launches Bart, a vintage LLM trained on pre-1931 text

Unbounded Labs built Bart as a deliberate experiment in grounding AI in historical knowledge. By restricting training data to pre-1931 English, they aim to see if an LLM can independently derive conclusions similar to those of past scientists, a question inspired by Demis Hassabis. The project cost only $800 over three months, highlighting that such research is accessible even on a small budget.

The team had to create new benchmarks because none existed for evaluating a model trained on vintage text. Their full account covers corpus sourcing, cleaning, every ablation, training runs, and post-training, offering a transparent look at the process. The quote "What I cannot create, I do not understand" underscores their motivation to build from scratch rather than fine-tune existing models.

The release of Bart is significant for AI research because it tackles a fundamental question: whether models can generate original ideas or merely predict the next token. While General Relativity was out of budget, the project suggests that exploring vintage domains could provide insights into AI's creative potential. The article and demo allow others to examine the approach and results directly.