EU AI Act OpenRAG: 933 legally structured chunks in SQLite

EU AI Act OpenRAG is a structured corpus of Regulation (EU) 2024/1689 released for researchers and developers working on RAG systems and legal NLP tasks. The core innovation is the chunking strategy: instead of sliding a window across characters, the corpus chunks the text at legal boundaries—one chunk per article paragraph, one per recital, one per Article 3 definition, and one per annex point. Chapter, section, and provision metadata are stored as separate fields rather than embedded in chunk text. This approach yields 933 chunks in total.

Each chunk is paired with a normalized 1024-dimensional embedding computed using BGE-M3, an embedding model. The entire corpus is packaged as a single SQLite database file for easy download and deployment. The corpus also includes exact links to EUR-Lex (the official EU law database), metadata on Article 113 (which specifies application dates), and labels designed with deliberate narrowness—textual classification is stored separately from broader regulatory-regime association, and ambiguous cases are explicitly marked NULL to avoid false positive matches.

The developer evaluated the corpus against the AI Act Evaluation Benchmark using a like-for-like whole-unit baseline. Two key metrics showed improvement: scenario article recall@20 (the ability to retrieve the correct article among the top 20 results) reached 0.541 with the structured approach versus 0.449 with the baseline; QA article hit@10 (whether the correct article appears in the top 10 results for a question) reached 0.927 versus 0.898. These gains demonstrate that structurally aware chunking improves both retrieval and question-answering performance on legal text.

The release of EU AI Act OpenRAG addresses a specific gap in legal AI infrastructure. Most RAG systems use sliding character windows to chunk documents, which breaks legal text at arbitrary points and loses the intentional structure that lawyers and regulators rely on for interpretation. By aligning chunks with the Regulation's formal structure—articles, recitals, definitions, and annexes—the corpus preserves the semantic and legal boundaries that matter for accurate retrieval.

The performance gains the developer measured are concrete: scenario article recall@20 improved from 0.449 to 0.541, and QA article hit@10 improved from 0.898 to 0.927. These gains suggest that structurally aware chunking helps both retrieval of relevant provisions and question-answering over legal text. The inclusion of metadata—chapter/section/provision taxonomy, EUR-Lex identifiers, and application dates—further supports downstream legal reasoning. The deliberate separation of direct textual labels from broader regulatory associations, with NULL entries for ambiguous cases, indicates an attempt to avoid false confidence in regulatory classification, a practical concern for systems that must cite or interpret rules.

No comments yet. Be the first to share your thoughts!