Metadata-Version: 2.4
Name: iil-ingest
Version: 0.1.0
Summary: Reusable Document Ingestion Package for IIL Platform (ADR-170)
Project-URL: Homepage, https://github.com/achimdehnert/iil-ingest
Project-URL: Repository, https://github.com/achimdehnert/iil-ingest
Project-URL: Issues, https://github.com/achimdehnert/iil-ingest/issues
Author-email: Achim Dehnert <achim@dehnert.dev>
License: MIT
Keywords: classifier,document,iil,iil-platform,ingestion,pdf
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Text Processing
Classifier: Typing :: Typed
Requires-Python: >=3.11
Provides-Extra: all
Requires-Dist: openpyxl>=3.1; extra == 'all'
Requires-Dist: pdf2image>=1.17; extra == 'all'
Requires-Dist: pdfplumber>=0.11; extra == 'all'
Requires-Dist: pytesseract>=0.3; extra == 'all'
Requires-Dist: python-docx>=1.1; extra == 'all'
Provides-Extra: dev
Requires-Dist: openpyxl>=3.1; extra == 'dev'
Requires-Dist: pdfplumber>=0.11; extra == 'dev'
Requires-Dist: pytesseract>=0.3; extra == 'dev'
Requires-Dist: pytest-cov>=5.0; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff>=0.4; extra == 'dev'
Provides-Extra: django
Requires-Dist: django>=5.0; extra == 'django'
Provides-Extra: docx
Requires-Dist: python-docx>=1.1; extra == 'docx'
Provides-Extra: excel
Requires-Dist: openpyxl>=3.1; extra == 'excel'
Provides-Extra: ocr
Requires-Dist: pdf2image>=1.17; extra == 'ocr'
Requires-Dist: pytesseract>=0.3; extra == 'ocr'
Provides-Extra: pdf
Requires-Dist: pdfplumber>=0.11; extra == 'pdf'
Description-Content-Type: text/markdown

# iil-ingest

> Reusable Document Ingestion Package for the IIL Platform — **ADR-170**

**Pattern:** iil-enrichment (ADR-169) — Pure Python Core, Protocol-based, optional Django integration

## Installation

```bash
pip install iil-ingest[pdf]        # PDF only
pip install iil-ingest[all]        # PDF + Excel + DOCX
pip install iil-ingest[all,django] # + Django mixins
```

## Quick Start

```python
from ingest import IngestPipeline, ProfileClassifier
from ingest.extractors.pdf import PDFExtractor
from ingest.profiles.german_hr import GERMAN_HR_PROFILES

pipeline = IngestPipeline(
    extractors=[PDFExtractor()],
    classifier=ProfileClassifier(GERMAN_HR_PROFILES),
)

with open("rechnung.pdf", "rb") as f:
    doc = pipeline.run(f.read(), filename="rechnung.pdf")

print(doc.doc_type)    # "RECHNUNG"
print(doc.confidence)  # "HIGH"
print(doc.score)       # 42.0
```

## Custom Profile

```python
from dataclasses import dataclass
from ingest import IngestPipeline, ProfileClassifier
from ingest.extractors.pdf import PDFExtractor

@dataclass
class SdsProfile:
    name = "SDS"
    patterns = [
        (r"sicherheitsdatenblatt", 10),
        (r"gefahrenhinweis|h\d{3}", 8),
        (r"reach", 6),
    ]
    min_score = 10

pipeline = IngestPipeline(
    extractors=[PDFExtractor()],
    classifier=ProfileClassifier([SdsProfile()]),
)
```

## Architecture

```
Upload -> iil-ingest (detect + extract + classify)
       -> iil-enrichment (optional enrichment)
       -> Django DB / Paperless (archive)
```

## Supported Formats

| Format | Extra | Extractor |
|--------|-------|-----------|
| PDF | `[pdf]` | `PDFExtractor` (pdfplumber) |
| Excel (.xlsx) | `[excel]` | `ExcelExtractor` (openpyxl) |
| CSV | -- | `CSVExtractor` (stdlib) |
| DOCX | `[docx]` | `DOCXExtractor` (python-docx) |

## Pipeline Flow

```
bytes + filename
  -> detect_mime()          # magic bytes + extension
  -> extractor.extract()    # text, tables, metadata
  -> classifier.classify()  # doc_type, confidence, score
  -> IngestedDocument
```

## References

- ADR-170: `platform/docs/adr/ADR-170-iil-ingest-document-ingestion-package.md`
- Pattern: iil-enrichment (ADR-169)
- Migrated from: `dms-hub/apps/benefits/classifier.py`, `dms-hub/apps/accounting/extractor.py`
