Metadata-Version: 2.5
Name: iil-enrichment
Version: 0.2.0
Summary: Enrichment Agent Pattern — bridge managed records with external knowledge sources (ADR-169)
Author-email: Achim Dehnert <achim@iil.gmbh>
License: MIT
Classifier: Development Status :: 3 - Alpha
Classifier: Framework :: Django :: 5.0
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.12
Provides-Extra: all
Requires-Dist: django>=5.0; extra == 'all'
Requires-Dist: hishel>=0.0.33; extra == 'all'
Requires-Dist: httpx>=0.27; extra == 'all'
Requires-Dist: pgvector>=0.3; extra == 'all'
Requires-Dist: pyrate-limiter>=3.6; extra == 'all'
Requires-Dist: tenacity>=9.0; extra == 'all'
Provides-Extra: dev
Requires-Dist: hishel>=0.0.33; extra == 'dev'
Requires-Dist: httpx>=0.27; extra == 'dev'
Requires-Dist: pyrate-limiter>=3.6; extra == 'dev'
Requires-Dist: pytest-cov>=5.0; extra == 'dev'
Requires-Dist: pytest-mock>=3.12; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: respx>=0.21; extra == 'dev'
Requires-Dist: ruff>=0.4; extra == 'dev'
Requires-Dist: tenacity>=9.0; extra == 'dev'
Provides-Extra: django
Requires-Dist: django>=5.0; extra == 'django'
Provides-Extra: gestis
Requires-Dist: hishel>=0.0.33; extra == 'gestis'
Requires-Dist: httpx>=0.27; extra == 'gestis'
Requires-Dist: pyrate-limiter>=3.6; extra == 'gestis'
Requires-Dist: tenacity>=9.0; extra == 'gestis'
Provides-Extra: http
Requires-Dist: hishel>=0.0.33; extra == 'http'
Requires-Dist: httpx>=0.27; extra == 'http'
Requires-Dist: pyrate-limiter>=3.6; extra == 'http'
Requires-Dist: tenacity>=9.0; extra == 'http'
Provides-Extra: pgvector
Requires-Dist: django>=5.0; extra == 'pgvector'
Requires-Dist: pgvector>=0.3; extra == 'pgvector'
Provides-Extra: pubchem
Requires-Dist: hishel>=0.0.33; extra == 'pubchem'
Requires-Dist: httpx>=0.27; extra == 'pubchem'
Requires-Dist: pyrate-limiter>=3.6; extra == 'pubchem'
Requires-Dist: tenacity>=9.0; extra == 'pubchem'
Description-Content-Type: text/markdown

# iil-enrichment

**Enrichment Agent Pattern** — bridge managed records with external knowledge sources.

> ADR-169: Generic pattern for enriching document-centric records (SDS, GBU, tenders, trades)
> with data from authoritative external APIs (GESTIS, PubChem, ECHA, market data, etc.)

## Installation

```bash
# Core only (no providers)
pip install iil-enrichment

# With GESTIS provider
pip install iil-enrichment[gestis]

# With Django integration
pip install iil-enrichment[django]

# Everything
pip install iil-enrichment[all]
```

## Quick Start

```python
from enrichment.registry import EnrichmentRegistry
from enrichment.providers.gestis import GESTISProvider
from enrichment.providers.pubchem import PubChemProvider

# Setup registry
registry = EnrichmentRegistry()
registry.register("substance", GESTISProvider())
registry.register("substance", PubChemProvider())

# Enrich a substance by CAS number
result = registry.enrich_merged("substance", "67-64-1")

# Access structured data
print(result.get("agw"))           # AGW from GESTIS
print(result.get("first_aid"))     # First aid from GESTIS
print(result.get("molecular_formula"))  # Formula from PubChem
print(result.confidence)           # 0.0–1.0
```

## Django Integration

```python
from enrichment.django.mixins import EnrichableModelMixin

class GlobalSdsRevision(EnrichableModelMixin, models.Model):
    substance = models.ForeignKey(...)

    def get_natural_key_for_enrichment(self):
        return self.substance.cas_number or self.product_name

    def get_enrichment_domain(self):
        return "sds"

# Usage
revision.run_enrichment()  # → fetches GESTIS + PubChem, stores in enrichment_data JSONB
```

## Architecture

```
┌──────────────┐     ┌──────────────────────┐
│ Managed      │     │ EnrichmentRegistry   │
│ Record       │────▶│                      │
│ (DMS-focus)  │     │  ┌──────────────┐   │
│              │     │  │ GESTISProv.  │   │
│ enrichment_  │◀────│  │ PubChemProv. │   │
│ data (JSONB) │     │  │ CustomProv.  │   │
└──────────────┘     │  └──────────────┘   │
                     └──────────────────────┘
```

## Built-in Providers

| Provider | Domain | Data |
|----------|--------|------|
| `GESTISProvider` | substance, sds | AGW, first aid, storage, transport, WGK, physical properties |
| `PubChemProvider` | substance, sds | Molecular data, GHS classification, H/P codes |

## Configuration via Environment

All settings have sensible defaults and can be overridden per environment:

| Env-Var                          | Default                     | Description                                |
|----------------------------------|-----------------------------|--------------------------------------------|
| `IIL_ENRICHMENT_TIMEOUT`         | `15`                        | HTTP timeout in seconds                    |
| `IIL_ENRICHMENT_MAX_RETRIES`     | `3`                         | Max retry attempts per request             |
| `IIL_ENRICHMENT_BACKOFF_INITIAL` | `0.5`                       | Initial backoff in seconds                 |
| `IIL_ENRICHMENT_BACKOFF_MAX`     | `8.0`                       | Max backoff in seconds                     |
| `IIL_ENRICHMENT_CACHE`           | `1`                         | Cache on (`1`) or off (`0`)                |
| `IIL_ENRICHMENT_CACHE_DIR`       | `./.cache/iil-enrichment/`  | Cache directory                            |
| `IIL_ENRICHMENT_RATE_LIMIT`      | `1`                         | Rate limit on (`1`) or off (`0`)           |
| `IIL_ENRICHMENT_USER_AGENT`      | `iil-enrichment/0.2 ...`   | User-Agent for external APIs               |
| `GESTIS_API_KEY`                 | (public demo key)           | Own DGUV key recommended for production    |

Or configure programmatically:

```python
from enrichment.config import HTTPDefaults
from enrichment.providers.gestis import GESTISProvider

defaults = HTTPDefaults(timeout_seconds=30, max_retries=5, cache_enabled=True)
provider = GESTISProvider(defaults=defaults)
```

## Custom Provider

```python
from enrichment.provider import EnrichmentProvider
from enrichment.types import EnrichmentResult, PropertyValue

class MyProvider:
    @property
    def name(self) -> str:
        return "MySource"

    @property
    def supported_domains(self) -> list[str]:
        return ["my_domain"]

    def can_enrich(self, domain: str, natural_key: str) -> bool:
        return domain == "my_domain" and bool(natural_key)

    def enrich(self, domain: str, natural_key: str) -> EnrichmentResult:
        # Fetch from your API...
        return EnrichmentResult(
            source=self.name,
            confidence=0.9,
            properties={"key": PropertyValue(value="data")},
            natural_key=natural_key,
        )
```

## Ecosystem

`iil-enrichment` ist die **Anreicherungs-Stufe** einer dreiteiligen Datenpipeline:

```
iil-ingest  →  iil-enrichment  →  iil-fieldprefill
(extract)      (enrich)           (prefill form fields)
```

Eingang ist ein *natural key* (z. B. CAS-Nummer, oft aus `iil-ingest`-Text extrahiert);
Ausgang `EnrichmentResult.properties` speist den Kontext für `iil-fieldprefill`.
Vollständiger Flow, reale Typen und ein lauffähiges End-to-End-Beispiel:
**[platform/docs/ecosystem/iil-data-pipeline.md](https://github.com/achimdehnert/platform/blob/main/docs/ecosystem/iil-data-pipeline.md)**.

## License

MIT
