Metadata-Version: 2.4
Name: BODtoJSON
Version: 2.4.0
Summary: Zero-dependency OAGIS BOD to Flattened JSON, NDJSON, and Schema Projection engine
Author: Niraj Kakodkar
License-Expression: MIT
Project-URL: Homepage, https://thekakodkar.github.io/BODtoJSON/
Project-URL: Repository, https://github.com/thekakodkar/BODtoJSON
Project-URL: Documentation, https://thekakodkar.github.io/BODtoJSON/
Project-URL: Changelog, https://github.com/thekakodkar/BODtoJSON/blob/master/CHANGELOG.md
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Information Technology
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Text Processing :: Markup :: XML
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: api
Requires-Dist: fastapi>=0.100.0; extra == "api"
Requires-Dist: uvicorn>=0.20.0; extra == "api"
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0; extra == "dev"
Requires-Dist: httpx>=0.24.0; extra == "dev"
Requires-Dist: fastapi>=0.100.0; extra == "dev"
Requires-Dist: ruff>=0.6; extra == "dev"
Requires-Dist: mypy>=1.8; extra == "dev"
Requires-Dist: build>=1.0; extra == "dev"
Requires-Dist: twine>=5.0; extra == "dev"
Dynamic: license-file

# BODtoJSON v2.4.0

**BODtoJSON** is a safe, enterprise-grade Python data liquidity engine designed to transform complex Infor OAGIS (XML) Business Object Documents into modern, AI-ready flattened JSON, streaming NDJSON, and declarative target schemas.

<p>
<a href="https://thekakodkar.github.io/BODtoJSON/"> BODtoJSON Library Documentation </a>
</p>
<center><img width="769" height="336" alt="BOD2JSon" src="https://github.com/user-attachments/assets/0b089179-f137-4d56-b717-85cffb20d622" /></center>

v2.4.0 restores a **zero-runtime-dependency** core that is safe enough for Infor ION Scripting: XML is parsed with a stdlib-only hardened loader, while `to_records()` / `project()`, single-pass flattening, and the optional FastAPI extra stay unchanged.

## 🚀 What’s New in v2.4.0 (vs. v1.0.0)

* **Zero-Dependency Safe Core:** No third-party runtime dependencies. XML is parsed with a stdlib-only hardened loader (`xml.parsers.expat` + `xml.etree.ElementTree`) that rejects DTD/entity declarations, XXE, billion-laughs and quadratic blowup, and enforces size and depth limits.

* **Infor ION Scripting Helpers (`to_records` & `project`):** Eliminates boilerplate XML namespace parsing, `json.loads()` line-splitting loops, and manual boolean coercion in ION scripts. Captures `ApplicationArea` and `DataArea/<Verb>` header metadata automatically (`include_header=True`) and maps records directly into target REST/API JSON schemas.

* **Explicit Dual-Pipeline Architecture:** Dedicated top-level API methods—`to_json` and `to_ndjson`—improve interface predictability and self-documentation.

* **Strict NDJSON Linearization:** Features compact serialization via `separators=(',', ':')` and forced line termination (`\n`). Multi-noun batch payloads inside `DataArea` are automatically unpacked and emitted as individual, single-line text records.

* **100% Backward Compatibility:** The original `convert()` function remains active at the package root, routing to `to_json()` while emitting a non-breaking `DeprecationWarning` scheduled for removal in **v3.0.0**.

* **Enterprise REST API Tier (`api.py`):** Added an optional, high-performance FastAPI microservice layer featuring zero-overhead streaming using raw `Response` classes and `/project` schema mapping.

## 📈 Architectural Comparison

| Feature | Legacy (v1.0.0) | **Enterprise Engine (v2.4.0)** | 
| ----- | ----- | ----- | 
| **Core Dependencies** | Relied on `xmltodict` | **None (Python stdlib only)** | 
| **Memory & Speed** | Double-pass XML $\rightarrow$ Dict $\rightarrow$ Flat Dict | **Single-pass ElementTree traversal (\~50% lower memory)** | 
| **Data Formats** | Standard Structured JSON string | **Standard JSON, Strict NDJSON, Native Dicts & Projected Schemas** | 
| **Infor ION Scripting** | Manual XML + JSON loop boilerplate | **Declarative `project()` with automatic header context** | 
| **Ingestion Target** | Application memory / NoSQL DBs | **Data Lakehouses (Snowflake/Databricks), Vector DBs & ION APIs** | 
| **System Delivery** | Library imports only | **Library + Optional FastAPI Microservice Tier (`[api]`)** | 

## 🛠️ Installation

### Core Library (Zero Runtime Dependencies)

Ideal for Infor ION Scripting, data-engineering pipelines, ETL workers, and air-gapped environments. `pip install BODtoJSON` pulls no third-party packages (`pip show BODtoJSON` lists no Requires). ION hosts need the CPython stdlib modules `xml.etree.ElementTree` and `xml.parsers.expat` (ElementTree is expat-backed on CPython):

```bash
pip install BODtoJSON
```

### With Enterprise REST API Tier

Includes `fastapi` and `uvicorn` for microservice deployment:

```bash
pip install "BODtoJSON[api]"
```

## 📂 Project Structure

```text
BODtoJSON/
├── src/
│   └── BODtoJSON/
│       ├── __init__.py      # Package version (2.4.0) & API exports
│       ├── safe_xml.py      # Stdlib-only hardened XML parser
│       ├── mapper.py        # Dual-Pipeline & Projection Engine
│       └── api.py           # Optional FastAPI Microservice Ingestion Layer
├── tests/
│   ├── data/                # Raw OAGIS XML Test Datasets (.xml)
│   ├── config.py            # Test Data Configuration
│   ├── conftest.py          # Pytest Dataset Loader Fixtures
│   ├── test_mapper.py       # Core Engine Unit Tests, ION Projection & Regression Suite
│   └── test_api.py          # API Endpoint Verification
├── scripts/
│   └── inspect_payload.py   # Multi-Profile Business Validation Utility
├── pyproject.toml           # Build, Optional Extras & Pytest Configurations
└── README.md
```

## 🧪 Validation & Testing

### 1. Automated Testing (QA Suite)

Run the standardized `pytest` execution engine:

```powershell
python -m pytest -v -p no:cacheprovider
```

### 2. Manual Payload Inspection (UAT Engine)

Verify transformation output profiles locally:

```powershell
python scripts/inspect_payload.py
```

*Toggle the `mode` parameter within `inspect_payload.py` between `"ndjson"`, `"json"`, `"project"`, and `"legacy"` to inspect outputs in `output_preview.json`.*

## 💻 Code Examples

### 1. Infor ION Scripting: Declarative Schema Projection (`project`)

*Eliminates manual XML namespace handling, `json.loads()` loops, and boolean boilerplate.*

```python
import json
from BODtoJSON import project

SCHEMA = {
    "externalSystemId": "company[{Sync_AccountingEntityID}];DIMENSION1;{ChartOfAccounts_BaseChartOfAccounts_GLNominalAccount}",
    "isActive": "true",
    "companyId": "company[3100]",
    "dimensionName": "DIMENSION1",
    "usableInPurchase": "true",
}

# Automatically merges DataArea/Sync header context with each flattened Noun record
items = project(input_var, verb="Sync", schema=SCHEMA, include_header=True)
output_var = json.dumps(items)
```

### 2. Native Python Records (`to_records`)

*Extracts a Python `list[dict]` directly with optional `ApplicationArea` and `Verb` header context.*

```python
from BODtoJSON import to_records

records = to_records(xml_input, verb="Sync", include_header=True)
for rec in records:
    print(rec["Sync_AccountingEntityID"], rec["PurchaseOrder_PurchaseOrderHeader_DisplayID"])
```

### 3. Modern Pipeline: Newline Delimited JSON (`to_ndjson`)

*Optimized for Data Lakehouses (Snowflake/Databricks) and Vector Database ingestion.*

```python
from BODtoJSON import to_ndjson

xml_input = """<SyncPurchaseOrder>...</SyncPurchaseOrder>"""

# Returns a dense, single-line string per record with zero internal spaces, terminated by \n
linearized_ndjson = to_ndjson(xml_input, verb="Sync")
```

### 4. Modern Pipeline: Standard Flattened JSON (`to_json`)

*Optimized for application-level consumption and document-store indexing.*

```python
from BODtoJSON import to_json

xml_input = """<SyncPurchaseOrder>...</SyncPurchaseOrder>"""

# Returns a standard flat JSON string
flattened_json = to_json(xml_input, verb="Sync")
```

### 5. Legacy Pipeline: Backward-Compatible Conversion (`convert`)

*Maintained for smooth migration of existing production pipelines.*

```python
from BODtoJSON import convert

xml_input = """<SyncPurchaseOrder>...</SyncPurchaseOrder>"""

# DEPRECATED: Scheduled for removal in v3.0.0.
# Internally routes to to_json() and emits a DeprecationWarning.
legacy_json = convert(xml_input, verb="Sync")
```

### 6. Running the Enterprise API Tier

Launch the microservice locally:

```powershell
uvicorn BODtoJSON.api:app --reload
```

Open `http://127.0.0.1:8000/docs` to test `/convert/ndjson`, `/convert/json`, and `/project` directly via Swagger UI.

## 🤝 Commercial Value

By normalizing complex ERP structures into safe, flat JSON, streaming NDJSON, and projected API payloads, **BODtoJSON v2.4.0** stays at zero third-party runtime dependencies and reduces AI, Lakehouse, and Infor ION integration overhead by up to 90%.

**Author:** Niraj Kakodkar

**License:** MIT
## 🔒 Security (XML parsing)

The core library never calls `xml.etree.ElementTree.fromstring`. Input is scanned for `DOCTYPE` / `ENTITY` declarations (including UTF-16/32 and BOM encodings) and then parsed with `xml.parsers.expat` handlers that forbid DTDs, entity declarations, and external entity references. Parameter-entity parsing is disabled. A default 10 MiB size cap and a default depth cap of 256 apply; both are configurable on `BODConverter`.

This is stricter than a default `defusedxml.ElementTree.fromstring` call (used in 2.3.1 with `forbid_dtd=False`). Literal `<!DOCTYPE` or `<!ENTITY` tokens in comments or CDATA are also rejected. Infor OAGIS BODs do not need DTDs.

The optional API (`pip install "BODtoJSON[api]"`) is unauthenticated and caps request bodies at 10 MiB (HTTP 413). Put it behind your own auth and a reverse-proxy size limit for production.

## Known Limitations

- Flattening is lossy and one-way. There is no JSON → BOD reverse mapping.
- `to_json()` returns only the **first** noun under `DataArea`. Use `to_records()` or `to_ndjson()` to get all nouns.
- The optional FastAPI extra is unauthenticated. Deploy it only on a trusted network or behind your own authentication/authorization.
- Mixed content (text + child elements) preserves the direct text as `_#text` where present; tail text after children is not specially handled beyond ElementTree defaults.
- Empty elements flatten to JSON `null`.
- Documents whose text, comments, or CDATA contain the literal tokens `<!DOCTYPE` or `<!ENTITY` are rejected.

