Researchers create SiPaKosa, a massive 9.25M-word corpus of Buddhist texts in Sinhala and Pali to advance language model research for underrepresented languages.

SiPaKosa contains approximately 786K sentences combining 16 historical Buddhist documents and the complete web-scraped Tripitaka canonical texts

The corpus was built using Google Document AI for high-quality OCR on historical manuscripts, followed by systematic web scraping and rigorous quality control

Evaluation of 10 pretrained language models showed proprietary models outperform open-source alternatives by 3-6 times, with perplexity scores ranging from 1.09 to 189.67

The organized language-specific subcorpora supports domain-adapted language model pretraining and enables historical language analysis for Buddhist texts