Metadata-Version: 2.5
Name: pydiggs
Version: 1.13.0
Summary: Offline validation for DIGGS geotechnical XML instances (XSD, dictionary, Schematron).
Project-URL: Homepage, https://github.com/xinp-hub/pydiggs
Project-URL: Documentation, https://xinp-hub.github.io/pydiggs/
Project-URL: Source, https://github.com/xinp-hub/pydiggs
Project-URL: Issues, https://github.com/xinp-hub/pydiggs/issues
Project-URL: Changelog, https://github.com/xinp-hub/pydiggs/blob/master/CHANGELOG.md
Author-email: Xin Peng <xin_peng@outlook.com>
Maintainer-email: Xin Peng <xin_peng@outlook.com>
License-Expression: Apache-2.0 AND MPL-2.0 AND MIT AND LicenseRef-Energistics-PLA AND LicenseRef-POSC-LA AND LicenseRef-OGC-2001 AND LicenseRef-EPSG-TOU AND LicenseRef-DIGGS-Unstated AND LicenseRef-OASIS
License-File: LICENSE
License-File: LICENSES/LicenseRef-DIGGS-Unstated.txt
License-File: LICENSES/LicenseRef-EPSG-TOU.txt
License-File: LICENSES/LicenseRef-Energistics-PLA.txt
License-File: LICENSES/LicenseRef-OASIS.txt
License-File: LICENSES/LicenseRef-OGC-2001.txt
License-File: LICENSES/LicenseRef-POSC-LA.txt
License-File: NOTICE
License-File: src/pydiggs/dictionaries/NOTICE
License-File: src/pydiggs/schemas/LICENSE
Keywords: DIGGS,XML,geotechnical,schema,validation
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering
Classifier: Typing :: Typed
Requires-Python: >=3.11
Requires-Dist: lxml>=6.1.3
Requires-Dist: rich>=13.9.4
Provides-Extra: frames
Requires-Dist: pandas>=2.2.3; extra == 'frames'
Provides-Extra: gis
Requires-Dist: geopandas>=1.1.2; extra == 'gis'
Requires-Dist: pandas>=2.2.3; extra == 'gis'
Requires-Dist: pyarrow>=23.0.1; extra == 'gis'
Requires-Dist: pyogrio>=0.10.0; extra == 'gis'
Requires-Dist: pyproj>=3.7.0; extra == 'gis'
Requires-Dist: shapely>=2.0.6; extra == 'gis'
Provides-Extra: mcp
Requires-Dist: mcp>=2.3.0; extra == 'mcp'
Requires-Dist: python-multipart>=0.0.31; extra == 'mcp'
Provides-Extra: polars
Requires-Dist: polars>=1.20.0; extra == 'polars'
Provides-Extra: server
Requires-Dist: fastapi>=0.146.0; extra == 'server'
Requires-Dist: pydantic>=2.12.0; extra == 'server'
Requires-Dist: python-multipart>=0.0.31; extra == 'server'
Requires-Dist: starlette>=1.6.0; extra == 'server'
Requires-Dist: uvicorn>=0.54.0; extra == 'server'
Description-Content-Type: text/markdown

# pyDIGGS

[![PyPI version](https://img.shields.io/pypi/v/pydiggs.svg)](https://pypi.python.org/pypi/pydiggs)
[![Python Versions](https://img.shields.io/pypi/pyversions/pydiggs)](https://pypi.org/project/pydiggs)
[![Downloads](https://static.pepy.tech/badge/pydiggs)](https://pepy.tech/project/pydiggs)
[![Downloads/Month](https://static.pepy.tech/badge/pydiggs/month)](https://pepy.tech/project/pydiggs)
[![Build Status](https://github.com/xinp-hub/pydiggs/actions/workflows/ci.yml/badge.svg)](https://github.com/xinp-hub/pydiggs/actions/workflows/ci.yml)
[![Documentation Status](https://github.com/xinp-hub/pydiggs/actions/workflows/docs.yml/badge.svg)](https://xinp-hub.github.io/pydiggs/)
[![GitHub Issues](https://img.shields.io/github/issues/xinp-hub/pydiggs.svg)](https://github.com/xinp-hub/pydiggs/issues)
[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)

A Python package for Data Interchange for Geotechnical and Geoenvironmental Specialists (DIGGS).

* Licensed under the Apache License 2.0; see [Licence and use](https://xinp-hub.github.io/pydiggs/license/)
  for redistribution, bundled third-party material and professional use
* Documentation: [https://xinp-hub.github.io/pydiggs](https://xinp-hub.github.io/pydiggs)
* GitHub: [https://github.com/xinp-hub/pydiggs](https://github.com/xinp-hub/pydiggs)
* PyPI: [https://pypi.org/project/pydiggs](https://pypi.org/project/pydiggs)
* Release updates: **Watch → Custom → Releases** on this repository, the
  [releases feed](https://github.com/xinp-hub/pydiggs/releases.atom), or email
  (see [Stay updated](https://xinp-hub.github.io/pydiggs/stay-updated/))

## Authors

* **Project lead:** Xin Peng (<xin_peng@outlook.com>)
* **Contributor:** Asitha Senanayake (<asitha.senanayake@utexas.edu>)

## Contributing

Contributions are welcome under the project's Apache-2.0 licence, with each commit signed
off (`git commit -s`) under the
[Developer Certificate of Origin](https://developercertificate.org/). See
[CONTRIBUTING.md](https://github.com/xinp-hub/pydiggs/blob/master/CONTRIBUTING.md).

## Features

* Validate DIGGS instance files against:
    * DIGGS XSD Schema (**2.5.a**, **2.6**, **3.0.0**) with namespace auto-detect
      (default profile **3.0.0** when the namespace is unknown)
    * DIGGS Standard dictionaries / codeSpace semantics (offline bundle from diggsml.org/def)
    * Schematron business rules (bundled DIGGS lxml-adapted rules by default)
    * Lightweight context checks (structure / SRS / xlink / dataBlock arity)
* Read what is in a file: `pydiggs.read()` returns boreholes, samples, tests and their
  result tables as Python objects and as JSON (diggs-model 1.0), the same shape for
  DIGGS 2.5.a, 2.6 and 3.x ([Reading files](https://xinp-hub.github.io/pydiggs/model/)).
  With the `frames`, `polars` and `gis` extras, `pydiggs.frames` gives pandas or
  polars data frames and `pydiggs.gis` gives GeoDataFrames, GeoPackage and
  GeoParquet files. For reading, the public modules are `pydiggs`, `pydiggs.model`,
  `pydiggs.crs`, `pydiggs.frames` and `pydiggs.gis`.
* AGS4: read, write and check AGS4 files against the AGS4 rules (`pydiggs.ags4`,
  `pydiggs ags4 check`), and convert AGS4 to DIGGS 3.0.0 and DIGGS to AGS4 (4.2 or
  4.1.1) with every loss reported (`pydiggs.convert`, `pydiggs convert`,
  `POST /v1/convert`; [AGS4 files](https://xinp-hub.github.io/pydiggs/ags4/),
  [AGS4 and DIGGS conversion](https://xinp-hub.github.io/pydiggs/convert/)). No AGS dictionary or
  example file is bundled, and pyDIGGS is not an AGS-approved tool.
* Compare two DIGGS files: `pydiggs diff old.xml new.xml` and `pydiggs.diff.compare()`
  list the boreholes, samples, tests and values that were added, removed or changed,
  whatever the `gml:id`s, the element order and the number formatting; a change of unit
  is always reported and nothing is converted
  ([Comparing two files](https://xinp-hub.github.io/pydiggs/diff/)).
* Flexible validation output:
    * Machine-readable findings with stable rule ids, lines, XPaths and `gml:id`s, as
      JSON (diggs-findings 1.2) or SARIF 2.1.0 (`pydiggs validate`, `validate()`)
    * Write validation errors to log files
    * Print validation errors directly to console
* Command-line interface (CLI) for easy integration
* Use it from any application:
    * Python: `validate()` for files, `validate_bytes()` for uploads and other
      in-memory data
    * Web apps: an HTTP API with OpenAPI 3.1 (the `server` extra, then
      `pydiggs serve --http`, or mount `pydiggs.server.create_router()` in FastAPI)
    * Containers: that HTTP API as a non-root, attested image for amd64 and arm64,
      `ghcr.io/xinp-hub/pydiggs:X.Y.Z` (no `latest` tag)
    * Desktop and native apps: a stdio JSON-RPC 2.0 sidecar (`pydiggs serve --stdio`)
      that any host language can drive
    * AI assistants: an MCP server with read-only tools to validate a document, explain
      rules, and read features, samples, tests and result tables (the `mcp` extra,
      then `pydiggs serve --mcp`)
    * Browsers: in a web page with Pyodide, so the file never leaves the user's machine
      (a dependency-free Web Worker example is in `examples/browser-worker`)

## Security

pyDIGGS treats every DIGGS file as untrusted input. All XML is parsed with external
entities, DTD loading and network access refused, so a file cannot make pyDIGGS read
other local files or contact a server. There are two exceptions:

* A `codeSpace` that points at a local dictionary (`file:///`, `./`, `../`). The DIGGS
  whitelist permits these, and pyDIGGS parses them with the same hardened settings.
* With `DictionarySemanticValidator(allow_network=True)`, which is off by default,
  pyDIGGS downloads dictionaries, but only from whitelisted http/https URLs,
  re-checked on every redirect, with time and size limits.

Schemas and Schematron rules are different: they are code, not data. A custom
`schema_path` or `schematron_path` runs with your permissions (Schematron is compiled to
XSLT, which can read local files), so only pass files you trust. The bundled DIGGS
schemas and rules make no file or network access.

Log files are written to the working directory and are never written through a
symlink. Vulnerabilities: please report them privately (see [SECURITY.md](SECURITY.md)).

Install from the PyPI wheels, which bundle libxml2, rather than building lxml against
an older system libxml2.

## Web Application

Not a Python user? Looking for leveraging the power of pyDIGGS in a modern web application? Check out:

* [DXplorer](https://www.dx-plorer.com)

## Quick Start

Install pydiggs (requires Python 3.11+), then validate a file. The
[Quickstart](https://xinp-hub.github.io/pydiggs/quickstart/) walks through it:
```bash
uv tool install pydiggs   # or: python -m pip install pydiggs
pydiggs validate BH-01.xml
```

Basic usage with Python. `uv tool install` keeps pydiggs in its own environment, so to
import it, add it to your project with `uv add pydiggs` (or pip) first:
```python
from pydiggs import detect_diggs_version, validator

# Profile auto-detected from the instance namespace (unknown → 3.0.0)
validation = validator("path/to/your/diggs_file.xml")

print(detect_diggs_version("path/to/your/diggs_file.xml"))  # "2.5.a", "2.6", or "3.0.0"

# Or pin a profile explicitly (Python API only)
validation = validator("path/to/your/diggs_file.xml", diggs_version="2.6")

# Schema validation
validation.schema_check()

# Dictionary validation (DIGGSml semantic checks; offline dictionaries)
validation.dictionary_check()

# Schematron validation (bundled DIGGS rules by default)
validation.schematron_check()

# Context checks
validation.context_check()

# Or supply a custom Schematron file
validation = validator("path/to/your/diggs_file.xml", schematron_path="path/to/schematron.sch")
validation.schematron_check()
```

Every check at once, as machine-readable findings:
```bash
# JSON (diggs-findings 1.2); exit 0 = no errors, 1 = errors, 2 = usage/unreadable input
pydiggs validate --format json "path/to/your/diggs_file.xml"

# A whole folder as SARIF 2.1.0 for code-scanning tools
pydiggs validate --format sarif --output pydiggs.sarif "path/to/folder"
```
```python
from pydiggs import validate

report = validate("path/to/your/diggs_file.xml")  # prints nothing, writes no files
print(report.ok, report.counts)
for finding in report.findings:
    print(finding.rule_id, finding.severity, finding.line, finding.xpath)
```
See [findings](https://xinp-hub.github.io/pydiggs/findings/) for the format, rule ids,
exit codes and privacy defaults (no snippets, no absolute paths).

Basic usage with CLI:
```bash
# Schema validation (auto-detects DIGGS 2.5.a / 2.6 / 3.0.0)
pydiggs schema_check "path/to/your/diggs_file.xml"

# Dictionary validation
pydiggs dictionary_check "path/to/your/diggs_file.xml"

# Schematron validation (bundled rules)
pydiggs schematron_check "path/to/your/diggs_file.xml"

# Context validation
pydiggs context_check "path/to/your/diggs_file.xml"

# Custom Schematron
pydiggs schematron_check "path/to/your/diggs_file.xml" --schematron_path "path/to/schematron.sch"
```

See [usage](https://xinp-hub.github.io/pydiggs/usage/) for `diggs_version` / `schema_path`
overrides, log output options, and [known limits](https://xinp-hub.github.io/pydiggs/usage/#known-limits)
(dictionary check 12 as WARNING, bundled Schematron subset, and more).

For more detailed information and advanced usage, please see the [documentation](https://xinp-hub.github.io/pydiggs).
