Metadata-Version: 2.4
Name: belgian-deduce
Version: 4.3.0
Summary: Rule-based de-identification for Belgian clinical text
License: GPL-3.0-only
License-File: LICENSE.md
Keywords: de-identification,clinical text,belgium,nlp
Author: Vincent Menger
Author-email: vmenger@protonmail.com
Maintainer: Stig Hellemans
Requires-Python: >=3.9,<4.0
Classifier: License :: OSI Approved :: GNU General Public License v3 (GPLv3)
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Software Development :: Pre-processors
Classifier: Topic :: Text Processing
Classifier: Topic :: Text Processing :: Linguistic
Requires-Dist: docdeid (==1.0.1)
Requires-Dist: frozendict (>=2.4.6,<3.0.0)
Requires-Dist: rapidfuzz (>=3.13.0,<4.0.0)
Requires-Dist: regex (>=2024.11.6,<2025.0.0)
Project-URL: Repository, https://github.com/stighellemans/belgian-deduce/
Description-Content-Type: text/markdown

[![black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)

# Belgian Deduce

<!-- start include in docs -->

`belgian_deduce` is a rule-based de-identification package for Belgian clinical text.
It ships as a standalone Python package, uses Belgian lookup data and defaults, and is
built on top of [docdeid](https://github.com/vmenger/docdeid).

* Remove names, places, institutions, dates, ages, identifiers, phone numbers, e-mail
  addresses, and URLs from Belgian medical text
* Tune behavior through config, lookup structures, and custom processors
* Use Belgian defaults for postal codes, phone numbers, and national register numbers

> `belgian_deduce` started from the original
> [deduce](https://github.com/vmenger/deduce) project. This repository now maintains
> its own package identity, configuration, documentation, and Belgian-specific
> defaults.

> De-identification is never perfect. Validate and adapt the package on your own data
> before using it in a critical environment.

## Citing

If you use `belgian_deduce`, cite the original DEDUCE paper for the underlying method
and reference this repository and version in your implementation notes:

[Menger, V.J., Scheepers, F., van Wijk, L.M., Spruit, M. (2017). DEDUCE: A pattern
matching method for automatic de-identification of Dutch medical text, Telematics and
Informatics, 2017, ISSN 0736-5853](http://www.sciencedirect.com/science/article/pii/S0736585316307365)

<!-- end include in docs -->

<!-- start getting started -->

## Installation

Install the latest release from PyPI:

```bash
pip install belgian-deduce
```

## Getting Started

```python
from belgian_deduce import Deduce

model = Deduce()

text = (
    "betreft: Jan Janssens, rijksregisternummer 85.07.30-033.28, patnr 000334433. "
    "De patient J. Janssens is 64 jaar oud en woont in Leuven. Hij werd op "
    "10 oktober 2018 door arts Peter de Smet ontslagen uit UZ Leuven. "
    "Voor nazorg kan hij worden bereikt via j.janssens.123@gmail.com of "
    "0470 12 34 56."
)

doc = model.deidentify(text)
print(doc.deidentified_text)
```

```text
betreft: [PERSON-1], rijksregisternummer [NATIONAL_REGISTER_NUMBER-1], patnr [ID-1]. De patient [PERSON-1] is [AGE-1] jaar oud en woont in [LOCATION-1]. Hij werd op [DATE-1] door arts [PERSON-2] ontslagen uit [HOSPITAL-1]. Voor nazorg kan hij worden bereikt via [EMAIL-1] of [PHONE_NUMBER-1].
```

If patient metadata is known, pass it explicitly:

```python
from belgian_deduce import Deduce, Person

model = Deduce()
patient = Person(first_names=["Jan"], initials="JJ", surname="Janssens")
doc = model.deidentify(text, metadata={"patient": patient})
```

Metadata can also be used for more than the primary patient. The pipeline supports:

* `persons`: one or more additional `Person` objects treated as regular people
* `addresses`: one or more `Address` objects that should be tagged as `location`
* `entities`: arbitrary exact metadata matches via `MetadataEntity`
* `Person.birth_date`, `Person.aliases`, and `Person.addresses`

```python
from datetime import date

from belgian_deduce import Address, Deduce, MetadataEntity, Person

deduce = Deduce()

metadata = {
    "patient": Person(
        first_names=["Jan"],
        surname="Jansen",
        birth_date=date(1980, 3, 12),
        addresses=[
            Address(
                street="Kerkstraat",
                house_number="12A",
                postal_code="9000",
                city="Gent",
            )
        ],
    ),
    "persons": [
        Person(
            first_names=["Peter"],
            surname="de Visser",
            aliases=["Dr. Peter de Visser"],
        )
    ],
    "entities": [
        MetadataEntity(text="UZ Gent", tag="hospital"),
        MetadataEntity(text="ABC-12345", tag="id"),
    ],
}

doc = deduce.deidentify(text, metadata=metadata)
```

Dates are replaced by placeholders by default. To pseudonymize detected dates while
preserving their format, enable date shifting and provide the actual shift through
metadata for each logical time-conserving block:

```python
deduce = Deduce(config={"redactor_date_strategy": "shift"})

for patient in patients:
    doc = deduce.deidentify(
        patient.text,
        metadata={"date_shift_days": patient.date_shift_days},
    )
```

Use a shift greater than 365 days. Shorter shifts are easier to reverse engineer from
weekday and holiday scheduling patterns, document creation patterns, and explicit
weekdays in the source text. Dense longitudinal records may require additional
safeguards even with a range longer than one year.

Prefer a separate date shift for each time-conserving block of text, such as a
patient-level or hospitalization-level block. Reusing one date shift for a whole
dataset increases the attack surface for re-identifying the original dates. For
this reason, pass `date_shift_days` in per-document metadata when possible instead
of setting one fixed `redactor_date_shift_days` value on a model that processes a
whole dataset.

Avoid this pattern for production datasets:

```python
deduce = Deduce(
    config={
        "redactor_date_strategy": "shift",
        "redactor_date_shift_days": 42,
    }
)
```

For repeated processing where you do not want to store explicit shift days, you can
derive a stable safe offset from metadata such as a birth date:

```python
deduce = Deduce(
    config={
        "redactor_date_strategy": "shift",
        "redactor_date_shift_seed_key": "birth_date",
    }
)
doc = deduce.deidentify(text, metadata={"birth_date": date(1980, 3, 12)})
```

### Birthdate age rendering

When a date is written with a degree prefix (`°12/01/2000`) it is treated as a
birthdate. If `metadata["document_creation_date"]` is present, the shifted
birthdate is rendered as age text relative to the shifted document creation
date; without that metadata key the shifted birthdate is reduced to a year or
year range instead. The redactor always uses age mode when it can, so removing
`document_creation_date` is the only way to force the year fallback here.

Age text gets coarser as the patient gets older: days for newborns, then weeks,
then months, then years. The same bands drive both kinds of source text — a
birthdate, which is measured against the shifted document creation date, and an
age already written as a duration, which is banded directly.

| Age                       | Replacement form                                                     | Examples                                     |
| ------------------------- | -------------------------------------------------------------------- | -------------------------------------------- |
| 28 days or less           | days                                                                  | `[1 dag oud]`, `[28 dagen oud]`              |
| 29 to 90 days             | weeks, plus the leftover days when it is not a whole number of weeks  | `[5 weken oud]`, `[12 weken, 6 dagen oud]`   |
| 91 days to under 6 months | months, plus the leftover whole weeks when there is at least one      | `[3 maanden oud]`, `[3 maanden, 1 week oud]` |
| 6 to 23 months            | months, never weeks                                                   | `[6 maanden oud]`, `[23 maanden oud]`        |
| 2 to 11 years             | years, plus the leftover months unless the age lands on the birthday  | `[2 jaar oud]`, `[11 jaar, 10 maanden oud]`  |
| 12 years and older        | years, never months                                                   | `[12 jaar oud]`, `[34 jaar oud]`             |

The first two boundaries are counted in whole days. From 91 days on, months and
years are completed calendar months and years counted from the birthdate, not
fixed day counts, and a day-of-month is clamped when the target month is
shorter, so a 29 February birthdate uses 28 February in non-leap years.

#### Ages already written in the text

When the source text already gives an age rather than a birthdate, it is banded
the same way, so the same real age reads the same however the source wrote it.
The phrasing is kept: text that already sits at its band comes back untouched,
adjectival forms stay adjectival, and English stays English.

| Source                | Replacement            | Why                                  |
| --------------------- | ---------------------- | ------------------------------------ |
| `12 jaar`             | `[12 jaar]`            | already at its band                  |
| `6-jarig`             | `[6-jarig]`            | already at its band, form kept       |
| `7 jaar 7 maand`      | `[7 jaar 7 maand]`     | already at its band                  |
| `18 weken`            | `[4 maanden]`          | past 90 days, so months              |
| `35 maanden 23 dagen` | `[2 jaar, 11 maanden]` | past 24 months, so years and months  |
| `2 weken`             | `[14 dagen]`           | 14 days sits inside the days band    |
| `1-jarige`            | `[12 maanden]`         | a one-year-old is rendered in months |
| `61,2 yrs`            | `[61 yrs]`             | fractional years drop to whole years |
| `18 wks`              | `[4 months]`           | English is kept                      |

An age needs no `document_creation_date`, because it is already a duration.
Where one is supplied it is used as the anchor, so an age and the equivalent
birthdate agree; otherwise a fixed internal anchor keeps the output
deterministic. The two can only differ for an expression sitting exactly on a
band edge, because calendar months vary in length: `3 maanden` is 89 to 92 days
and the weeks band ends at 90.

Gestational age is not an age under the annotation guideline — `39w` in
`Aterme baby (39w)` is not annotated. If one is labelled `Age_Birthdate` anyway,
it is banded like an age and reads as the baby's age.

#### Worked age examples

Every row uses a shifted document creation date of `2026-05-14`. The birthdates
are shifted values too, so the rendered age is the patient's true age.

| Shifted birthdate | Age                 | Replacement                 |
| ----------------- | ------------------- | --------------------------- |
| `2026-05-13`      | 1 day               | `[1 dag oud]`               |
| `2026-04-17`      | 27 days             | `[27 dagen oud]`            |
| `2026-04-16`      | 28 days             | `[28 dagen oud]`            |
| `2026-04-15`      | 29 days             | `[4 weken, 1 dag oud]`      |
| `2026-03-15`      | 60 days             | `[8 weken, 4 dagen oud]`    |
| `2026-02-28`      | 75 days             | `[10 weken, 5 dagen oud]`   |
| `2026-02-13`      | 90 days             | `[12 weken, 6 dagen oud]`   |
| `2026-02-12`      | 91 days             | `[3 maanden oud]`           |
| `2026-02-03`      | 3 months, 11 days   | `[3 maanden, 1 week oud]`   |
| `2025-11-15`      | 5 months, 29 days   | `[5 maanden, 4 weken oud]`  |
| `2025-11-13`      | 6 months            | `[6 maanden oud]`           |
| `2025-05-14`      | 1 year              | `[12 maanden oud]`          |
| `2024-06-14`      | 23 months           | `[23 maanden oud]`          |
| `2024-05-14`      | 2 years             | `[2 jaar oud]`              |
| `2023-09-14`      | 2 years, 8 months   | `[2 jaar, 8 maanden oud]`   |
| `2014-06-20`      | 11 years, 10 months | `[11 jaar, 10 maanden oud]` |
| `2014-05-14`      | 12 years            | `[12 jaar oud]`             |
| `1992-01-03`      | 34 years            | `[34 jaar oud]`             |

Three rows in that table regularly surprise people:

- `2025-05-14` is exactly one year old and renders as `[12 maanden oud]`, not
  `[1 jaar oud]`. Months are used all the way up to 23 months.
- `2026-02-13` is 90 days old and still renders in weeks; one day older it
  becomes `[3 maanden oud]`. The weeks band runs on a day count, not on
  calendar months.
- `2026-02-03` renders `1 week`, counted from the 11 days left over after three
  whole months, not from its 100-day total. Below 91 days the weeks are counted
  from the total instead, which is why `2026-03-15` reads `8 weken, 4 dagen oud`.

#### Intervals, fallback, and precision

An approximate birthdate such as `zomer 2013` covers a range of dates, so the
age is rendered at both ends of the shifted range. Equal ends collapse to a
single value; different ends are joined with a slash, as in
`[11 jaar, 11 maanden oud/12 jaar oud]`.

If the shifted birthdate falls after the shifted document creation date, no age
can be computed and the span falls back to the shifted year or year range.

The first three bands are day- and week-precise. For a neonate,
`[3 dagen oud]` next to a document date pins the birthdate to the day within the
shifted timeline, so it stays safe only while the shift itself stays secret and
is not reused across a dataset. Omit `document_creation_date` for those
records when that precision is not wanted; an age written in the text is
banded either way.

French-speaking notes can be handled through the same API. A practical path is to
provide metadata for names, birth dates, addresses, and institutions:

```python
from datetime import date

from belgian_deduce import Address, Deduce, MetadataEntity, Person

deduce = Deduce()

text = (
    "Patient Jean Dupont, né le 12 mars 1980, habite Rue de la Loi 12, "
    "1000 Bruxelles. Sophie Martin consulte à Hôpital Erasme."
)

metadata = {
    "patient": Person(
        first_names=["Jean"],
        surname="Dupont",
        birth_date=date(1980, 3, 12),
        addresses=[
            Address(
                street="Rue de la Loi",
                house_number="12",
                postal_code="1000",
                city="Bruxelles",
            )
        ],
    ),
    "persons": [Person(first_names=["Sophie"], surname="Martin")],
    "entities": [MetadataEntity(text="Hôpital Erasme", tag="hospital")],
}

doc = deduce.deidentify(text, metadata=metadata)
print(doc.deidentified_text)
```

```text
Patient [PATIENT], né le [DATE-1], habite [LOCATION-1]. [PERSON-1] consulte à [HOSPITAL-1].
```

<!-- end getting started -->

## Documentation

The project documentation lives in [docs/source/tutorial.md](docs/source/tutorial.md)
and [docs/source/migrating.md](docs/source/migrating.md).

## Contributing

Contribution guidance is available in [CONTRIBUTING.md](CONTRIBUTING.md).

## Versions

* `4.3.0` - Integrated production span post-processing and labels, added
  metadata-driven known values, and hardened date and age pseudonymization
* `4.2.0` - Added configurable date shifting with safety warnings and guidance
  for patient-level or hospitalization-level shifts
* `4.1.0` - Improved francophone coverage in Wallonia and Brussels, especially
  for healthcare institutions and locations
* `4.0.1` - Polished the published package page and stabilized the tag-driven
  release workflow
* `4.0.0` - First standalone `belgian_deduce` release with Belgian defaults and
  independent docs/tooling
* Earlier entries in [CHANGELOG.md](CHANGELOG.md) predate the standalone release and
  are preserved for provenance

## Authors

* Vincent Menger - original DEDUCE implementation
* Stig Hellemans - Belgian standalone package and maintenance

## License

This project is licensed under the GNU General Public License v3.0, following
the licence of DEDUCE. See [LICENSE.md](LICENSE.md).

