Metadata-Version: 2.4
Name: eurlex-corpus
Version: 0.2.1
Summary: EU financial regulation as structured, cross-referenced, multilingual data
Author: Naveen
License: MIT
Project-URL: Homepage, https://github.com/chenjigaram/eurlex-corpus
Project-URL: Source, https://github.com/chenjigaram/eurlex-corpus
Keywords: eur-lex,regulation,compliance,regtech,mifid,legal,nlp
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Legal Industry
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Text Processing :: Linguistic
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: requests>=2.28
Requires-Dist: beautifulsoup4>=4.11
Provides-Extra: dev
Requires-Dist: pytest>=7; extra == "dev"
Requires-Dist: ruff>=0.4; extra == "dev"
Requires-Dist: mcp>=2.0; extra == "dev"
Provides-Extra: mcp
Requires-Dist: mcp>=2.0; extra == "mcp"
Dynamic: license-file

# eurlex-corpus

EU financial regulation as structured, cross-referenced, multilingual data.

[![ci](https://github.com/Chenjigaram/eurlex-corpus/actions/workflows/ci.yml/badge.svg)](https://github.com/Chenjigaram/eurlex-corpus/actions/workflows/ci.yml) [![pypi](https://img.shields.io/pypi/v/eurlex-corpus.svg)](https://pypi.org/project/eurlex-corpus/)

**[What I found building it →](https://chenjigaram.github.io/eurlex-corpus/)**

Eight EU financial acts, split to the paragraph, in all 24 official languages, with the
alignment verified rather than assumed. The corpus is the point; this package is what
builds it.

**[chenjigaram/eu-financial-regulation-aligned](https://huggingface.co/datasets/chenjigaram/eu-financial-regulation-aligned)**
— 56,838 rows, 2,442 provisions, CC BY 4.0.

### Related work

Getting text out of EUR-Lex is well covered, and both of these do more of it than this
package does:

- [`kevin91nl/eurlex`](https://github.com/kevin91nl/eurlex) parses any CELEX document
  into a pandas DataFrame.
- [`tseidl/eurlex-builder`](https://github.com/tseidl/eurlex-builder) is a research
  pipeline producing DuckDB and Parquet tables, handling six HTML eras plus scanned
  PDFs, granularity down to the lettered point, and document relations taken from
  Cellar metadata. If you need breadth, reach for that one first.

What neither produces is a corpus of the **official translations aligned to each other**.
`eurlex-builder` normalises to English, machine-translating non-English sources — the
right call for policy analysis, but it means the multilingual text is model output rather
than the human translation the Union actually published. This keeps all 24 versions and
aligns them exactly, which is what makes the result usable as a parallel corpus.

The other narrower difference: relations here resolve to the **target article**
(`Article 16(7) of Regulation 600/2014`), where Cellar metadata only records
document-to-document ties. Theirs is more authoritative; this one is finer-grained.

```bash
pip install eurlex-corpus
```

```python
from eurlex_corpus import load

mifid = load("MiFID2", lang="en")

mifid.article("16").title                    # 'Organisational requirements'
mifid.article("16").paragraph(7).text        # 'Records shall include the recording of telephone...'
mifid.article("16").url                      # deep link to the paragraph on EUR-Lex
mifid.article("16").citation                 # '32014L0065 Article 16'
mifid.article("16").heading                  # chapter/section chain + title

mifid.article("16").translations["nl"].paragraph(7).text
# 'Het bijhouden van gegevens omvat het opnemen van telefoongesprekken...'

mifid.article("16").references
# [Reference('Article 17' -> 32014L0065/17),
#  Reference('Article 4 of Directive 2006/73/EC' -> 32006L0073/4),
#  Reference('Regulation (EU) No 600/2014' -> 32014R0600/None), ...]
```

Documents are cached on first use under `~/.cache/eurlex`, so everything after the
first call is local.

## Coverage

| Key | Act | CELEX | Articles | Paragraphs |
|---|---|---|---:|---:|
| `mifid2` | MiFID II — Directive 2014/65/EU | 32014L0065 | 97 | 406 |
| `mifir` | MiFIR — Regulation (EU) No 600/2014 | 32014R0600 | 55 | 231 |
| `priips` | PRIIPs — Regulation (EU) No 1286/2014 | 32014R1286 | 34 | 89 |
| `ucits` | UCITS — Directive 2009/65/EC | 32009L0065 | 119 | 319 |
| `aifmd` | AIFMD — Directive 2011/61/EU | 32011L0061 | 71 | 363 |
| `dora` | DORA — Regulation (EU) 2022/2554 | 32022R2554 | 64 | 254 |
| `mica` | MiCA — Regulation (EU) 2023/1114 | 32023R1114 | 149 | 732 |
| `sfdr` | SFDR — Regulation (EU) 2019/2088 | 32019R2088 | 20 | 48 |

609 articles, 2,442 numbered paragraphs. Any other act works too if you pass its CELEX
id directly — `load("32016R0679")` — you just do not get the short name.

Consolidated texts work the same way: `load("02014L0065-20240328")`.

## The alignment is exact, not approximate

EUR-Lex assigns the same structural ids to every language version of an act. Article 16
is `art_16` in all of them, and its seventh paragraph is `016.007` whether you asked for
English, Dutch, German or French.

That means aligning translations needs no sentence matching and no heuristics. Checked
across all eight acts in all 24 languages — 181 act-language pairs:

| | Pairs | |
|---|---|---|
| Article ids identical to English | **181/181** | 100% |
| Article *and* paragraph ids identical | **179/181** | 98.9% |

```bash
python tools/check_alignment.py
```

The two exceptions are UCITS and AIFMD in Croatian. Croatia joined the EU in 2013, so
acts adopted before then appear in Croatian as a special edition using the older EUR-Lex
layout, which has no structural ids at all. Those parse to the right 119 and 71 articles
but carry no paragraph ids, so they can be aligned by article and not below it.

MiFID II, UCITS and AIFMD have no Irish version — the Irish language derogation ran
until 2022.

```python
en = load("dora", lang="en").article("11").paragraph(1).text
de = load("dora", lang="de").article("11").paragraph(1).text
```

Cellar serves all 24 official EU languages, and `lang=` accepts any of them.

## References work in every language

An EU act is numbered identically in all 24 versions, so the citation itself is
language-independent — only the word in front of it changes, and it inflects:
`Verordnung`, `Verordening`, `nařízení`, `asetuksen`, `рeгламент`. Extraction keys off
the number shape and treats the surrounding word as a hint rather than a requirement.

Because the article ids align exactly, that extractor can be checked without any
annotation: the same article in Dutch must cite the same acts as in English, and any
disagreement is a bug. Across MiFID II and DORA — 161 articles per language:

```bash
python tools/check_languages.py
```

| | Languages |
|---|---|
| 100% | cs, el, en, es, et, fr, nl, pt, ro, sv |
| 98–99% | bg, da, de, fi, hr, it, lt, mt, pl, sk, sl |
| 95–97% | ga, lv |
| below | hu (86%) |

Mean 98.5%, with 23 of 24 languages at or above 95%. Hungarian is the outlier: it
writes `1093/2010/EU rendelet`, putting the act word after the number and the number
before the year, which collides with the directive form used everywhere else.

## The citation graph

References only get you outbound edges. The useful question is usually the reverse one —
*what cites this?* — and EUR-Lex does not answer it.

```python
import eurlex_corpus

g = eurlex.graph()                       # all eight acts
len(g)                                   # 3662 citations, 184 act-to-act edges

g.inbound("32014L0065", "16")            # what cites MiFID II Article 16
g.outbound("32014R0600", "26")           # what MiFIR Article 26 cites
g.most_cited(5, external_only=True)      # most cited from outside their own act
g.dangling()                             # 83 acts cited but not loaded
```

The most-cited provisions across these eight acts, counting only citations from a
different act, are not the famous ones:

| Provision | Cited by other acts |
|---|---:|
| ESMA Regulation Article 19 — binding mediation | 15 |
| ESMA Regulation Article 15 — implementing technical standards | 14 |
| ESMA Regulation Article 16 — guidelines and recommendations | 10 |
| EBA Regulation Article 16 — guidelines and recommendations | 7 |

EU financial regulation leans on the supervisor's rule-making powers more than on any
substantive provision.

```bash
eurlex-corpus graph --top 10
eurlex-corpus graph --cited-by mifid2:16
```

## Use it from an AI assistant

The package ships an MCP server, so an assistant can look EU law up instead of
recalling it. Every answer carries the CELEX id, article number and a deep link.

```bash
pip install "eurlex-corpus[mcp]"
```

Add it to Claude Desktop's `claude_desktop_config.json`:

```json
{
  "mcpServers": {
    "eurlex": {
      "command": "eurlex-corpus-mcp"
    }
  }
}
```

Six tools: `list_acts`, `get_article`, `get_paragraph`, `search_articles`,
`compare_languages` and `who_cites`.

The server is told, in its own instructions, not to answer EU regulatory questions from
memory. That is the point of it — a model that quotes `32014L0065 Article 16(7)` and
links to it can be checked, and one that recalls the gist cannot.

`compare_languages` is the tool the alignment work pays for: ask for the same provision
in `en,nl,de` and you get the three official texts, not three translations of one of them.

## Command line

```bash
eurlex-corpus list                                  # known instruments
eurlex-corpus fetch                                 # cache all eight, English
eurlex-corpus fetch mifid2 dora --lang en --lang nl
eurlex-corpus show mifid2 16 --paragraph 7
eurlex-corpus show mifid2 16 --json
```

## What it does not do

- No search or retrieval. This is the data layer; ranking is your problem.
- Recitals, annexes and tables are not split out yet — article bodies only.
- Article-level references (`Article 16(7)` and its 23 translations) are extracted
  but not measured; only the act-level citations above are verified.
- Enumerations are expanded ("Articles 10 to 13" counts as four citations), but only
  where the connector is a recognised range word and the span is under 40 articles.
- MiFID II has no Irish version — the Irish derogation ran until 2022, so acts from
  before then are not available in `ga`.
- Not every act titles its articles. PRIIPs and UCITS give none at all, so `.title`
  is empty for all 153 of their articles. That is what the Official Journal says, not
  a parsing gap — use `.heading`, which falls back to the enclosing chapter and
  section titles.
- Cross-references are resolved by pattern, not by an official citation graph.
  Internal references and numbered EU acts resolve reliably; prose like
  "the Directive referred to in the preceding paragraph" does not.

## Rebuilding the dataset

```bash
python tools/build_dataset.py     # rebuild it
python tools/check_alignment.py   # verify the alignment claim
```

## Licence and attribution

The code is MIT. The texts are not mine to license:

> Source: EUR-Lex, © European Union. Reused under Commission Decision 2011/833/EU;
> editorial content is licensed CC BY 4.0.

Available as `eurlex.ATTRIBUTION`. Only the authentic printed Official Journal has legal
value — do not use this for anything where that distinction matters.

## Development

```bash
pip install -e ".[dev]"
pytest
ruff check .
```

Tests run offline against committed HTML fixtures.
