Metadata-Version: 2.4
Name: docgun
Version: 0.1.0
Summary: Deterministic document ingestion for PDF, DOCX, Excel, and CSV with quality-aware recovery.
Project-URL: Homepage, https://github.com/Arkay92/DocGun
Project-URL: Repository, https://github.com/Arkay92/DocGun
Project-URL: Issues, https://github.com/Arkay92/DocGun/issues
Project-URL: Changelog, https://github.com/Arkay92/DocGun/blob/main/CHANGELOG.md
Author: Arkay92
License-Expression: MIT
License-File: LICENSE
Keywords: csv,document-ingestion,docx,ocr,pdf,rag,xlsx
Classifier: Development Status :: 3 - Alpha
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Typing :: Typed
Requires-Python: >=3.11
Requires-Dist: defusedxml
Requires-Dist: fastapi
Requires-Dist: openpyxl
Requires-Dist: opentelemetry-api
Requires-Dist: orjson
Requires-Dist: pandas
Requires-Dist: pyarrow
Requires-Dist: pydantic>=2
Requires-Dist: pymupdf
Requires-Dist: python-docx
Requires-Dist: structlog
Requires-Dist: tenacity
Requires-Dist: uvicorn[standard]
Provides-Extra: dev
Requires-Dist: build>=1.0; extra == 'dev'
Requires-Dist: pytest; extra == 'dev'
Requires-Dist: twine>=5.0; extra == 'dev'
Provides-Extra: full
Requires-Dist: docling; extra == 'full'
Requires-Dist: polars; extra == 'full'
Requires-Dist: python-calamine; extra == 'full'
Requires-Dist: python-magic; extra == 'full'
Description-Content-Type: text/markdown

# DocGun

<p align="center">
  Quality-aware document ingestion for Python and RAG pipelines.
</p>

<p align="center">
  <img width="256" height="256" alt="DocGun logo" src="https://raw.githubusercontent.com/Arkay92/DocGun/refs/heads/main/DocGun.png" />
</p>

<p align="center">
  <a href="https://github.com/Arkay92/DocGun/actions/workflows/publish.yml"><img alt="Publish" src="https://github.com/Arkay92/DocGun/actions/workflows/publish.yml/badge.svg" /></a>
  <a href="https://pypi.org/project/docgun/"><img alt="PyPI" src="https://img.shields.io/pypi/v/docgun.svg" /></a>
  <img alt="Python" src="https://img.shields.io/pypi/pyversions/docgun.svg" />
  <img alt="Downloads" src="https://img.shields.io/pypi/dm/docgun.svg" />
  <img alt="License" src="https://img.shields.io/pypi/l/docgun.svg" />
</p>

> **Package name:** `docgun`  
> **Project name:** DocGun  
> **Install:** `pip install docgun`  
> **Import:** `from docgun import ingest`

**DocGun** turns PDF, DOCX, Excel, and CSV files into a consistent typed document model. It combines deterministic parsing with extraction scoring, page-aware fallback, security limits, and explicit recovery states so incomplete scans do not silently look successful.

- Native PDF text, coordinates, tables, and page provenance with PyMuPDF.
- Ordered DOCX headings, paragraphs, lists, and tables.
- Bounded XLSX/XLSM and batched CSV extraction.
- Page-level Docling fallback with structured elements and provenance.
- Injectable OCR/VLM recovery for only the pages that need it.
- Measurable confidence reports and structured recovery attempts.
- ZIP/Open XML validation, encrypted-PDF detection, and ingestion limits.
- Python API, CLI, and FastAPI upload route.

---

## Before / After

```python
from docgun import ingest

state = ingest("mixed-report.pdf")

print(state.status)
print(state.quality.confidence)
print(len(state.document.elements))
```

Example output:

```text
completed
0.9142
47
```

When a scan cannot be recovered, DocGun returns `recovery_required` with page-level markers and diagnostics instead of presenting an empty page as successfully ingested.

---

## Why Quality-Aware Ingestion?

Document parsers can return technically valid but incomplete output: a scanned page may yield only a footer, a fallback may omit pages, or a table detector may consume nearby text. DocGun makes those failure modes visible:

```text
Input document
  -> Validate size, signature, archive safety, and encryption
  -> Inspect pages or workbook dimensions
  -> Route to a deterministic native parser
  -> Score content, structure, tables, and provenance
  -> Compare page-level fallback candidates
  -> Recover only weak pages with Docling or an OCR/VLM backend
  -> Return typed elements, status, quality, warnings, and diagnostics
```

Pipeline statuses are `completed`, `partially_completed`, `recovery_required`, and `failed`.

---

## Install

```bash
pip install docgun
```

For Docling and the complete fallback toolset:

```bash
pip install "docgun[full]"
```

For development:

```bash
pip install -e ".[dev,full]"
pytest -q
python -m build
```

---

## Quick Start

### Ingest a Document

```python
from docgun import ingest

state = ingest("annual-report.pdf")

for element in state.document.elements:
    print(element.type, element.text, element.location)
```

### Inspect Recovery Diagnostics

```python
for attempt in state.recovery_attempts:
    print(attempt.action, attempt.status, attempt.error_message)
```

### Configure Page-Level Vision Recovery

Implement `extract_batch(rendered_pages)` in your OCR or VLM adapter and inject it into the pipeline:

```python
from docgun.agents.recovery import RecoveryEngine
from docgun.agents.vision import VisionPageRecovery
from docgun.pipeline.graph import ingest

class MyVisionBackend:
    def extract_batch(self, pages):
        return vision_client.extract(pages)

engine = RecoveryEngine(VisionPageRecovery(MyVisionBackend()))
state = ingest("scan.pdf", recovery_engine=engine)
```

Each backend result may contain `text`, `type`, `table`, `bbox`, and `confidence` fields.

---

## CLI

Ingest a file and print the complete pipeline state as JSON:

```bash
docgun report.pdf
```

Print a compact summary:

```bash
docgun report.pdf --summary
```

---

## Supported Formats

| Format | Native parser | Recovery |
|---|---|---|
| PDF | PyMuPDF | Docling, then configured page-level OCR/VLM |
| DOCX | python-docx | Explicit low-confidence status |
| XLSX/XLSM | openpyxl | Explicit low-confidence status |
| CSV | PyArrow | Explicit low-confidence status |

---

## Architecture

```text
src/docgun/
  __init__.py          # Public API
  cli.py               # Command-line interface
  agents/              # Inspection, routing, confidence, quality, recovery
  api/                 # FastAPI upload route
  chunking/            # Hierarchical and spreadsheet chunk helpers
  domain/              # Typed elements, inspections, and errors
  parsers/             # PDF, Docling, DOCX, Excel, and CSV parsers
  pipeline/            # Orchestration state and policies
  security/            # File limits and format validation
  storage/             # Document and vector storage interfaces
tests/                 # Unit and regression tests
.github/workflows/     # CI and trusted PyPI publishing
CHANGELOG.md            # Release history
pyproject.toml          # Package metadata and dependencies
```

---

## FastAPI

The bundled route streams uploads to a bounded temporary file, maps ingestion errors to appropriate HTTP responses, and removes the temporary file after success or failure.

```python
from fastapi import FastAPI
from docgun.api.routes import router

app = FastAPI()
app.include_router(router, prefix="/documents")
```

---

## Development

```bash
pip install -e ".[dev,full]"
python -m pytest
python -m build
python -m twine check dist/*
```

---

## License

MIT

---

## Contributing

Contributions are welcome. Please include a representative document or minimal generated fixture, the expected elements, and a regression test for parser or recovery changes.

---

## Citation

```bibtex
@software{DocGun2026,
  title={DocGun: Quality-Aware Document Ingestion for Python},
  author={Arkay92},
  url={https://github.com/Arkay92/DocGun},
  year={2026},
  version={0.1.0},
}
```

---

## Acknowledgments

- [PyMuPDF](https://pymupdf.readthedocs.io/) for fast native PDF extraction and rendering.
- [Docling](https://github.com/docling-project/docling) for layout-aware document conversion.
- [python-docx](https://python-docx.readthedocs.io/) and [openpyxl](https://openpyxl.readthedocs.io/) for Office document parsing.
- [Apache Arrow](https://arrow.apache.org/) for batched CSV ingestion.
