eurlex-corpus

EU financial regulation as structured, cross-referenced, multilingual data.

What I found building it →

EUR-Lex publishes every act as HTML behind a SPARQL endpoint. Getting the text out is solved — eurlex parses any CELEX document into a DataFrame and does it well. What is not solved is everything after that: the same act in 24 languages, aligned paragraph by paragraph, and the citation graph that connects the acts to each other. That is what this is.

pip install eurlex-corpus

from eurlex_corpus import load

mifid = load("MiFID2", lang="en")

mifid.article("16").title                    # 'Organisational requirements'
mifid.article("16").paragraph(7).text        # 'Records shall include the recording of telephone...'
mifid.article("16").url                      # deep link to the paragraph on EUR-Lex
mifid.article("16").citation                 # '32014L0065 Article 16'
mifid.article("16").heading                  # chapter/section chain + title

mifid.article("16").translations["nl"].paragraph(7).text
# 'Het bijhouden van gegevens omvat het opnemen van telefoongesprekken...'

mifid.article("16").references
# [Reference('Article 17' -> 32014L0065/17),
#  Reference('Article 4 of Directive 2006/73/EC' -> 32006L0073/4),
#  Reference('Regulation (EU) No 600/2014' -> 32014R0600/None), ...]

Documents are cached on first use under ~/.cache/eurlex, so everything after the first call is local.

Coverage

609 articles, 2,442 numbered paragraphs. Any other act works too if you pass its CELEX id directly — load("32016R0679") — you just do not get the short name.

Consolidated texts work the same way: load("02014L0065-20240328").

The alignment is exact, not approximate

EUR-Lex assigns the same structural ids to every language version of an act. Article 16 is art_16 in all of them, and its seventh paragraph is 016.007 whether you asked for English, Dutch, German or French.

That means aligning translations needs no sentence matching and no heuristics. Checked across all eight acts in all 24 languages — 181 act-language pairs:

python tools/check_alignment.py

The two exceptions are UCITS and AIFMD in Croatian. Croatia joined the EU in 2013, so acts adopted before then appear in Croatian as a special edition using the older EUR-Lex layout, which has no structural ids at all. Those parse to the right 119 and 71 articles but carry no paragraph ids, so they can be aligned by article and not below it.

MiFID II, UCITS and AIFMD have no Irish version — the Irish language derogation ran until 2022.

en = load("dora", lang="en").article("11").paragraph(1).text
de = load("dora", lang="de").article("11").paragraph(1).text

Cellar serves all 24 official EU languages, and lang= accepts any of them.

References work in every language

An EU act is numbered identically in all 24 versions, so the citation itself is language-independent — only the word in front of it changes, and it inflects: Verordnung, Verordening, nařízení, asetuksen, рeгламент. Extraction keys off the number shape and treats the surrounding word as a hint rather than a requirement.

Because the article ids align exactly, that extractor can be checked without any annotation: the same article in Dutch must cite the same acts as in English, and any disagreement is a bug. Across MiFID II and DORA — 161 articles per language:

python tools/check_languages.py

Mean 98.5%, with 23 of 24 languages at or above 95%. Hungarian is the outlier: it writes 1093/2010/EU rendelet, putting the act word after the number and the number before the year, which collides with the directive form used everywhere else.

The citation graph

References only get you outbound edges. The useful question is usually the reverse one — what cites this? — and EUR-Lex does not answer it.

import eurlex_corpus

g = eurlex.graph()                       # all eight acts
len(g)                                   # 3662 citations, 184 act-to-act edges

g.inbound("32014L0065", "16")            # what cites MiFID II Article 16
g.outbound("32014R0600", "26")           # what MiFIR Article 26 cites
g.most_cited(5, external_only=True)      # most cited from outside their own act
g.dangling()                             # 83 acts cited but not loaded

The most-cited provisions across these eight acts, counting only citations from a different act, are not the famous ones:

EU financial regulation leans on the supervisor's rule-making powers more than on any substantive provision.

eurlex-corpus graph --top 10
eurlex-corpus graph --cited-by mifid2:16

Command line

eurlex-corpus list                                  # known instruments
eurlex-corpus fetch                                 # cache all eight, English
eurlex-corpus fetch mifid2 dora --lang en --lang nl
eurlex-corpus show mifid2 16 --paragraph 7
eurlex-corpus show mifid2 16 --json

What it does not do

• No search or retrieval. This is the data layer; ranking is your problem.
• Recitals, annexes and tables are not split out yet — article bodies only.
• Article-level references (Article 16(7) and its 23 translations) are extracted but not measured; only the act-level citations above are verified.
• Enumerations are expanded ("Articles 10 to 13" counts as four citations), but only where the connector is a recognised range word and the span is under 40 articles.
• MiFID II has no Irish version — the Irish derogation ran until 2022, so acts from before then are not available in ga.
• Not every act titles its articles. PRIIPs and UCITS give none at all, so .title is empty for all 153 of their articles. That is what the Official Journal says, not a parsing gap — use .heading, which falls back to the enclosing chapter and section titles.
• Cross-references are resolved by pattern, not by an official citation graph. Internal references and numbered EU acts resolve reliably; prose like "the Directive referred to in the preceding paragraph" does not.

The dataset

The aligned corpus is published as a dataset in its own right — 56,838 rows, 2,442 provisions across 24 languages, CC BY 4.0:

chenjigaram/eu-financial-regulation-aligned

python tools/build_dataset.py     # rebuild it
python tools/check_alignment.py   # verify the alignment claim

Licence and attribution

The code is MIT. The texts are not mine to license:

Source: EUR-Lex, © European Union. Reused under Commission Decision 2011/833/EU; editorial content is licensed CC BY 4.0.

Available as eurlex.ATTRIBUTION. Only the authentic printed Official Journal has legal value — do not use this for anything where that distinction matters.

Development

pip install -e ".[dev]"
pytest
ruff check .

Tests run offline against committed HTML fixtures.