Metadata-Version: 2.4
Name: pycharter
Version: 0.0.68
Summary: Runtime data contract enforcement for Python: one Pydantic-native contract validates API objects, Kafka streams, and ETL batches in-process — with quality scoring, a versioned contract registry, and ODCS round-trip.
Author-email: Optophi <contact@optophi.com>
License-Expression: Apache-2.0
Project-URL: Changelog, https://github.com/optophi/pycharter/blob/develop/CHANGELOG.md
Project-URL: Homepage, https://github.com/optophi/pycharter
Project-URL: Documentation, https://optophi.github.io/pycharter/
Project-URL: Repository, https://github.com/optophi/pycharter
Project-URL: Issues, https://github.com/optophi/pycharter/issues
Project-URL: Contributing, https://github.com/optophi/pycharter/blob/main/CONTRIBUTING.md
Keywords: data-contract,data-contracts,runtime-validation,data-quality,data-governance,json-schema,pydantic,odcs,open-data-contract-standard,kafka,streaming,etl,validation,schema-registry,schema
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Topic :: Software Development :: Quality Assurance
Classifier: Topic :: Database
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: NOTICE
Requires-Dist: pydantic<3.0,>=2.11.0
Requires-Dist: jsonschema>=4.26.0
Requires-Dist: pandas>=2.0.0
Requires-Dist: numpy>=1.24.0
Requires-Dist: pyyaml>=6.0.0
Requires-Dist: sqlalchemy[asyncio]<3.0,>=2.0.0
Requires-Dist: alembic<2.0,>=1.13.0
Requires-Dist: jsonata-python>=0.6.0
Requires-Dist: httpx<1.0,>=0.24.0
Requires-Dist: networkx>=3.0
Provides-Extra: server
Requires-Dist: fastapi<0.144,>=0.136.3; extra == "server"
Requires-Dist: uvicorn[standard]<1.0,>=0.24.0; extra == "server"
Requires-Dist: starlette>=1.2.1; extra == "server"
Requires-Dist: pydantic-settings<3.0,>=2.14.2; extra == "server"
Requires-Dist: python-multipart>=0.0.6; extra == "server"
Requires-Dist: PyJWT<3.0,>=2.8.0; extra == "server"
Requires-Dist: httpx<1.0,>=0.24.0; extra == "server"
Requires-Dist: aiofiles>=25.1.0; extra == "server"
Requires-Dist: prometheus-client>=0.19.0; extra == "server"
Requires-Dist: psycopg[binary]>=3.3.4; extra == "server"
Provides-Extra: pipeline
Requires-Dist: sshtunnel>=0.4.0; extra == "pipeline"
Requires-Dist: boto3>=1.26.0; extra == "pipeline"
Requires-Dist: google-cloud-storage>=3.12.0; extra == "pipeline"
Requires-Dist: azure-storage-blob>=12.0.0; extra == "pipeline"
Requires-Dist: openpyxl>=3.1.5; extra == "pipeline"
Requires-Dist: lxml>=4.9.0; extra == "pipeline"
Requires-Dist: pyarrow>=14.0.0; extra == "pipeline"
Requires-Dist: pymongo>=4.0.0; extra == "pipeline"
Requires-Dist: openlineage-python>=1.0.0; extra == "pipeline"
Provides-Extra: streaming
Requires-Dist: websockets>=12.0; extra == "streaming"
Requires-Dist: watchfiles>=0.21.0; extra == "streaming"
Requires-Dist: prometheus_client>=0.19.0; extra == "streaming"
Requires-Dist: aiohttp>=3.9.0; extra == "streaming"
Requires-Dist: aiokafka>=0.9.0; extra == "streaming"
Requires-Dist: aio-pika>=9.0.0; extra == "streaming"
Requires-Dist: redis>=5.0.0; extra == "streaming"
Requires-Dist: python-socketio>=5.11.0; extra == "streaming"
Provides-Extra: semantic
Requires-Dist: linkml>=1.7.0; extra == "semantic"
Requires-Dist: linkml-runtime>=1.7.0; extra == "semantic"
Requires-Dist: rdflib>=7.0.0; extra == "semantic"
Requires-Dist: owlrl>=6.0.0; extra == "semantic"
Requires-Dist: neo4j<7.0.0,>=6.2.0; extra == "semantic"
Provides-Extra: extraction
Requires-Dist: tree-sitter<1.0.0,>=0.25.2; extra == "extraction"
Requires-Dist: tree-sitter-python<1.0.0,>=0.23.0; extra == "extraction"
Requires-Dist: tree-sitter-javascript<1.0.0,>=0.23.0; extra == "extraction"
Requires-Dist: tree-sitter-typescript<1.0.0,>=0.23.0; extra == "extraction"
Requires-Dist: tree-sitter-go<1.0.0,>=0.23.0; extra == "extraction"
Requires-Dist: tree-sitter-rust<1.0.0,>=0.24.2; extra == "extraction"
Requires-Dist: tree-sitter-java<1.0.0,>=0.23.5; extra == "extraction"
Requires-Dist: markdown-it-py<5.0.0,>=4.2.0; extra == "extraction"
Requires-Dist: pypdf<8.0.0,>=4.0.0; extra == "extraction"
Requires-Dist: sqlglot>=25.0; extra == "extraction"
Provides-Extra: ai
Requires-Dist: anthropic>=0.34.0; extra == "ai"
Requires-Dist: openai>=2.44.0; extra == "ai"
Requires-Dist: mcp<3,>=1.0; extra == "ai"
Provides-Extra: otel
Requires-Dist: opentelemetry-api<2,>=1.20; extra == "otel"
Provides-Extra: cli
Requires-Dist: rich<16.0,>=13.0.0; extra == "cli"
Provides-Extra: docs
Requires-Dist: mkdocs<2,>=1.5.3; extra == "docs"
Requires-Dist: mkdocs-material[imaging]>=9.7.6; extra == "docs"
Requires-Dist: mkdocstrings[python]>=0.24.0; extra == "docs"
Requires-Dist: pymdown-extensions>=10.0.0; extra == "docs"
Requires-Dist: mike>=2.0.0; extra == "docs"
Provides-Extra: airflow
Requires-Dist: apache-airflow<4,>=3.2.2; extra == "airflow"
Provides-Extra: dagster
Requires-Dist: dagster>=1.7; extra == "dagster"
Provides-Extra: prefect
Requires-Dist: prefect<4,>=3.0; extra == "prefect"
Requires-Dist: SQLAlchemy<2.1,>=2.0; extra == "prefect"
Provides-Extra: postgres
Requires-Dist: asyncpg>=0.29; extra == "postgres"
Requires-Dist: psycopg[binary]>=3.3.4; extra == "postgres"
Provides-Extra: reference
Requires-Dist: pycharter[postgres]; extra == "reference"
Requires-Dist: aiosqlite<1,>=0.22; extra == "reference"
Provides-Extra: all
Requires-Dist: pycharter[ai,cli,docs,extraction,otel,pipeline,postgres,reference,semantic,server,streaming]; extra == "all"
Provides-Extra: ci
Requires-Dist: pycharter[extraction,pipeline,postgres,prefect,reference,semantic,server,streaming]; extra == "ci"
Requires-Dist: setuptools>=77.0; extra == "ci"
Requires-Dist: mcp<3,>=1.0; extra == "ci"
Requires-Dist: testcontainers[postgres]>=4.14.2; extra == "ci"
Requires-Dist: testcontainers[kafka]>=4.14.2; extra == "ci"
Requires-Dist: pytest>=7.0.0; extra == "ci"
Requires-Dist: pytest-asyncio>=0.21.0; extra == "ci"
Requires-Dist: pytest-cov>=4.0.0; extra == "ci"
Requires-Dist: pytest-timeout>=2.1.0; extra == "ci"
Requires-Dist: pytest-xdist>=3.5.0; extra == "ci"
Requires-Dist: pytest-randomly>=3.15.0; extra == "ci"
Requires-Dist: ruff>=0.15.20; extra == "ci"
Requires-Dist: mypy>=1.0.0; extra == "ci"
Requires-Dist: types-PyYAML>=6.0.0; extra == "ci"
Requires-Dist: requests>=2.32.3; extra == "ci"
Requires-Dist: pre-commit>=3.0.0; extra == "ci"
Requires-Dist: mkdocs<2,>=1.5.3; extra == "ci"
Requires-Dist: mkdocs-material[imaging]>=9.7.6; extra == "ci"
Requires-Dist: mkdocstrings[python]>=0.24.0; extra == "ci"
Requires-Dist: pymdown-extensions>=10.0.0; extra == "ci"
Provides-Extra: dev
Requires-Dist: pycharter[all,prefect]; extra == "dev"
Requires-Dist: setuptools>=77.0; extra == "dev"
Requires-Dist: requests>=2.32.3; extra == "dev"
Requires-Dist: pytest>=7.0.0; extra == "dev"
Requires-Dist: pytest-asyncio>=0.21.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
Requires-Dist: pytest-timeout>=2.1.0; extra == "dev"
Requires-Dist: pytest-xdist>=3.5.0; extra == "dev"
Requires-Dist: pytest-randomly>=3.15.0; extra == "dev"
Requires-Dist: pytest-benchmark>=4.0.0; extra == "dev"
Requires-Dist: testcontainers[kafka]>=4.14.2; extra == "dev"
Requires-Dist: testcontainers[postgres]>=4.14.2; extra == "dev"
Requires-Dist: opentelemetry-sdk<2,>=1.20; extra == "dev"
Requires-Dist: ruff>=0.15.20; extra == "dev"
Requires-Dist: mypy>=1.0.0; extra == "dev"
Requires-Dist: types-PyYAML>=6.0.0; extra == "dev"
Requires-Dist: pre-commit>=3.0.0; extra == "dev"
Requires-Dist: build>=0.10.0; extra == "dev"
Requires-Dist: twine>=4.0.0; extra == "dev"
Requires-Dist: marimo>=0.9.0; extra == "dev"
Requires-Dist: vulture>=2.11; extra == "dev"
Requires-Dist: deptry>=0.20; extra == "dev"
Requires-Dist: pip-audit>=2.7; extra == "dev"
Dynamic: license-file

# PyCharter

## Work management

In this repository, **backlog** means the [Optophi — Delivery Project](https://github.com/orgs/optophi/projects/1).
The **PyCharter backlog** is its [repository-filtered view](https://github.com/orgs/optophi/projects/1/views/24), using `Record kind: Work`. Reference records are historical context.
[GitHub Issues](https://github.com/optophi/pycharter/issues) hold scope, acceptance criteria, dependencies and evidence;
Project fields hold current priority and delivery status. `Backlog` is a status
within that same Project, not a separate task list.

Do not maintain a parallel Markdown or Linear backlog. Historical sources and
legacy-ID mappings are retained through the [migration archive](https://github.com/optophi/pycharter/blob/develop/docs/archive/github-migration-2026-10-30/README.md);
they are provenance only. When adding work to the Project, set `Record kind: Work`.
Retained proposals, program boards and status tables are design
references or dated history; their presence does not authorize implementation.
Search existing Issues and follow migration mappings to the canonical Issue before
creating work. Link advancing PRs and keep local qualification, merge, release and
deployment evidence separate. Technical specifications remain in their owning documents.


**[Python users](docs/python/index.md)** · **[Coding agents](docs/agents/index.md)**

These source docs include develop features. See the Python guide for the
published-release boundary and a pinned offline quickstart.

**Define good data once. Check it wherever it flows.**

PyCharter turns data requirements into executable, versioned contracts. Use the
same rules to validate application records, check a batch, guard a pipeline
step, or verify an agent's output. When data fails, get an explanation you can
act on.

Start with one Python call. Grow toward a reliable data feed with reviewed
pipelines, inspectable delivery and a clear recovery path.

For Python developers checking records before an application consumes them,
the smallest useful workflow is a local validator or file check. The
[first-result walkthrough](https://optophi.com/docs/pycharter/getting-started/five-minutes/)
uses one order contract to accept data, explain a rejection and check the
corrected record.

[Get started](https://optophi.com/docs/pycharter/getting-started/five-minutes/) ·
[Build a data feed](https://optophi.com/docs/pycharter/getting-started/reliable-data-feed/) ·
[The complete product journey](https://optophi.com/docs/pycharter/getting-started/product-journey/) ·
[Documentation](https://optophi.com/docs/pycharter/)

[![Python 3.11+](https://img.shields.io/badge/python-3.11+-blue.svg)](https://www.python.org/downloads/)
[![License: Apache-2.0](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](LICENSE)

## Validate a record in 30 seconds

```bash
python -m pip install pycharter
```

The source core requires Python 3.11+ and Pydantic `>=2.11.0,<3.0`. When upgrading,
update any older explicit Pydantic pin to that supported range.

```python
from pycharter import Validator

validator = Validator.from_dict(
    {
        "type": "object",
        "version": "1.0.0",
        "properties": {
            "name": {"type": "string"},
            "age": {"type": "integer"},
        },
        "required": ["name", "age"],
    }
)

result = validator.validate({"name": "Alice", "age": 30})
print(result.is_valid)  # True

result = validator.validate({"name": "Bob", "age": "N/A"})
print(result.errors[0])  # Column 'age': expected integer, found string 'N/A'
```

This runs locally without a server, database, account or API key. By default,
compatible values such as numeric strings are coerced before validation; use
`strict_types=True` when constructing the validator to require strict types.
Load a saved contract with `Validator.from_file("contract.yaml")` when you want
the same requirements in another application or tool.

Prefer the shell? Create a contract document and a tiny local input file.
The Python factory above accepts a raw JSON Schema. For new contract files,
use the native v2 envelope shown in the first-result guide. Supported v1 files
remain loadable:

```bash
cat > contract.yaml <<'YAML'
api_version: pycharter.io/v2
kind: DataContract
metadata:
  name: person_record
  version: "1.0.0"
spec:
  json_schema:
    type: object
    version: "1.0.0"
    properties:
      name: {type: string}
      age: {type: integer}
    required: [name, age]
YAML
printf '{"name":"Alice","age":30}\n' > data.jsonl
pycharter validate contract.yaml data.jsonl
```

This reports one valid record and exits 0. Run the check again to repeat it.
To inspect a deliberate failure:

```bash
printf '{"name":"Bob","age":"N/A"}\n' > data.jsonl
pycharter validate contract.yaml data.jsonl
```

This exits 1 and identifies row 0 and column `age`. Restore the valid input above
and rerun the same command. This basic path needs no database or running service.

The [first-result guide](https://optophi.com/docs/pycharter/getting-started/five-minutes/)
provides complete commands, expected counts and exit codes for Python and the
file CLI. Its
[checked-in inputs](https://github.com/optophi/pycharter/tree/develop/examples/getting_started/)
include the deliberate failure and its correction; the guide also creates these files
without a clone. The exercise uses the published `0.0.67` package. Repository
examples are separate from installed package assets. The README examples also
run in the source wheel's minimum/current dependency gates.

## Two ways to start

| Your task | Start here | What you get |
| --- | --- | --- |
| **Check my data** | [Validate records](https://optophi.com/docs/pycharter/getting-started/five-minutes/), a [CSV](https://optophi.com/docs/pycharter/cookbook/validate-a-csv/) or a [DataFrame](https://optophi.com/docs/pycharter/cookbook/validate-a-dataframe/) | Accepted data and actionable failures against explicit requirements |
| **Create a reliable data feed** | [Run a contract-bearing local pipeline](https://optophi.com/docs/pycharter/getting-started/reliable-data-feed/) | A source, an enforced contract, an output you can inspect and a repeatable failure/recovery exercise |

You can keep contracts in your repository. Add a shared registry, services or
an external scheduler when your workflow needs them.

## What changes when the contract is executable

**Requirements travel with the data.** A contract combines structure, permitted
conversions, validation rules and optional quality or business metadata. People
can review the file; Python can enforce it.

**Failures become specific.** Find the record, field and failed requirement.
Choose a supported fail or quarantine policy at the enforcement boundary.
An inferred contract is a draft to review, not proof that the sample captured
every valid case.

**Pipelines and requirements stay connected.** Declare extraction,
transformation and loading in YAML, and attach a contract to the steps that
must enforce it. A contract stored beside a pipeline does not enforce itself.
[Try the local pipeline](https://optophi.com/docs/pycharter/getting-started/reliable-data-feed/).

**Agents work against checks you control.** Give an agent the relevant schema
and context, validate its candidate, and review changes to the requirements.
The validator supplies the verdict; the agent supplies a proposed answer or
repair. Contract conformance checks declared requirements, not factual truth.

**Operation has a clear destination.** The complete experience connects first
validation, verified delivery, scheduling, independent health and recovery.
The [product journey](https://optophi.com/docs/pycharter/getting-started/product-journey/) describes that
end state; [capability status](https://optophi.com/docs/pycharter/getting-started/capabilities/) distinguishes
available features, bounded references and work still being completed.

## Use the interface that fits your work

| Interface | Best for | Guide |
| --- | --- | --- |
| **Python** | Embed validation or run a pipeline inside your application | [Validator](https://optophi.com/docs/pycharter/api/validator/) · [Pipeline](https://optophi.com/docs/pycharter/api/pipeline/) |
| **CLI** | Check files, review changes and run versioned pipelines | [File validation](https://optophi.com/docs/pycharter/cookbook/validate-from-the-shell/) · [CLI reference](https://optophi.com/docs/pycharter/api/cli/) |
| **HTTP and UI** | Integrate applications and inspect contracts, pipelines and runs | [API and UI](https://optophi.com/docs/pycharter/tutorials/api-and-ui/) |
| **MCP and agents** | Discover contracts, prepare context and verify candidate records | [Agent guide](https://optophi.com/docs/pycharter/guides/pycharter-for-ai-agents/) · [MCP](https://optophi.com/docs/pycharter/guides/agent-integration-mcp/) |

The product direction is one coherent workflow across these interfaces. Use the
[capability guide](https://optophi.com/docs/pycharter/getting-started/capabilities/) for the current scope of
each surface; a configured MCP endpoint or UI does not itself provide scheduling.

## From an application check to an operated data feed

1. **Describe acceptable data.** Start from a sample, existing schema or contract.
   Review meaning, required fields and failure policy.
2. **Verify a bounded example.** Inspect accepted and rejected data before
   introducing live source credentials or destination changes.
3. **Deliver and inspect.** Enforce the contract at the selected boundary and
   inspect the actual output and execution result.
4. **Operate and recover.** Add scheduling, durable state and independent
   monitoring in a qualified deployment profile.

The [bounded FMP reference](https://optophi.com/docs/pycharter/guides/fmp-reference/) demonstrates the
operational direction: exact coverage checks, PostgreSQL publication receipts,
safe reruns and an independent monitor. It currently requires a reviewed source
checkout; the public `0.0.67` package cannot run that reference. Its fixture
uses synthetic data. Live access, release packaging and recurring production
operation have separate qualification gates.

For continuous data, the [streaming engine](https://optophi.com/docs/pycharter/guides/streaming-overview/)
applies contracts record by record with rollout modes and rejected-record
routing. Streaming remains experimental; delivery guarantees depend on the
selected broker, sink and mode.

## Add capabilities as you need them

```bash
python -m pip install pycharter                     # Local validation
python -m pip install "pycharter[pipeline,postgres]" # Additional connectors and Postgres drivers
python -m pip install "pycharter[server]"            # API, UI host and worker dependencies
python -m pip install "pycharter[ai]"                # MCP and agent integration dependencies
```

These install dependencies. Connection setup, credentials, service operation
and scheduling are explicit choices. See [Installation](https://optophi.com/docs/pycharter/getting-started/installation/)
for streaming, semantic and orchestrator extras.

## Minimal example (contract + pipeline + check)

This example is qualified against the source wheel. For public `0.0.67`, follow
the [file-pipeline installation note](https://optophi.com/docs/pycharter/getting-started/installation/#run-a-local-file-pipeline)
to install `sqlalchemy[asyncio]` before running it.

```bash
# 1) Define a contract
cat > contract.yaml <<'YAML'
api_version: pycharter.io/v2
kind: DataContract
metadata: { name: demo_contract, version: "1.0.0" }
spec:
  json_schema:
    type: object
    version: "1.0.0"
    required: [email, amount]
    properties:
      email:  { type: string, format: email }
      amount: { type: number, minimum: 0 }
YAML

# 2) Prepare a tiny input file
printf '{"email":"a@x.io","amount":10}\n{"email":"bad","amount":-1}\n' > input.jsonl

# 3) A file->file pipeline that copies the data (no network)
mkdir -p pipeline && cat > pipeline/pipeline.yaml <<'YAML'
api_version: pycharter.io/v2
kind: Pipeline
metadata: { name: copy_demo, version: "1.0.0" }
spec:
  variables: { IN: input.jsonl, OUT: output.jsonl }
  steps:
    - id: read
      type: extract
      source: { type: file, path: "${IN}", format: jsonl }
    - id: write
      type: load
      input: read
      target: { type: file, path: "${OUT}", format: jsonl, mode: overwrite }
YAML

# 4) Run the pipeline, then validate its output against the contract
pycharter pipeline run ./pipeline --watch
pycharter validate contract.yaml output.jsonl
```

The pipeline copies both rows and exits 0. The separate validation command
intentionally exits 1: row 1 has an invalid `email` and a negative `amount`.
This example checks the copied output; it does not attach enforcement to the
pipeline steps. Rerunning overwrites the output with the same two input rows.

To enforce the contract during delivery, continue with the
[local data-feed guide](https://optophi.com/docs/pycharter/getting-started/reliable-data-feed/).

## Keep your requirements portable

Use YAML/JSON contracts, version them with your code, and exchange the supported
ODCS subset with other tools. Shared vocabularies and field bindings add business
meaning when you need it. Start with the smallest useful contract and introduce
those capabilities progressively.

- [Contract formats and versioning](https://optophi.com/docs/pycharter/reference/artifact-format-versioning/)
- [ODCS import/export and supported semantics](https://optophi.com/docs/pycharter/guides/odcs-and-interop/)
- [Vocabulary and field meaning](https://optophi.com/docs/pycharter/guides/concepts/)
- [Where PyCharter fits in your stack](https://optophi.com/docs/pycharter/getting-started/comparisons/)

## Documentation and contribution

[Read the documentation](https://optophi.com/docs/pycharter/) for task guides,
API references and operating instructions. The [cookbook](https://optophi.com/docs/pycharter/cookbook/index/)
contains focused recipes for applications, files, streams and quality checks.

For development, follow [CONTRIBUTING.md](CONTRIBUTING.md),
[ARCHITECTURE.md](ARCHITECTURE.md) and [AGENTS.md](AGENTS.md).
Read [CHANGELOG.md](CHANGELOG.md) for release evidence, source-only Unreleased
changes and upgrade guidance.
Report security issues privately under [SECURITY.md](SECURITY.md).
Community expectations are in [CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md).

Licensed under the Apache License, Version 2.0. See [LICENSE](LICENSE).
