Metadata-Version: 2.4
Name: pipelantic
Version: 0.4.0
Summary: Typed, contract-driven data pipeline modeling for Python.
Project-URL: Homepage, https://github.com/eddiethedean/pipelantic
Project-URL: Documentation, https://github.com/eddiethedean/pipelantic/tree/main/docs
Project-URL: Repository, https://github.com/eddiethedean/pipelantic
Project-URL: Issues, https://github.com/eddiethedean/pipelantic/issues
Project-URL: Changelog, https://github.com/eddiethedean/pipelantic/blob/main/CHANGELOG.md
Author-email: Odo Matthews <odosmatthews@gmail.com>
License-Expression: MIT
License-File: LICENSE
Keywords: data-contracts,etl,pipelines,pydantic,typed
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Typing :: Typed
Requires-Python: >=3.11
Requires-Dist: anyio<5,>=4
Requires-Dist: click<9,>=8
Requires-Dist: contractmodel>=0.1.2
Requires-Dist: dpcs<1,>=0.13
Requires-Dist: dtcs<1,>=0.11
Requires-Dist: packaging>=24
Requires-Dist: pydantic<3,>=2.12
Requires-Dist: typer<1,>=0.15
Description-Content-Type: text/markdown

![Pipelantic banner](https://raw.githubusercontent.com/eddiethedean/pipelantic/main/docs/theme/assets/pipelantic-banner.png)

# Pipelantic

[![Documentation Status](https://readthedocs.org/projects/pipelantic/badge/?version=latest)](https://pipelantic.readthedocs.io/en/latest/?badge=latest)

Define, validate, plan, and locally run contract-driven data pipelines with
typed Python.

> Define data, transformations, and pipelines with typed Python classes.
> Validate and plan them once. Execute them through interchangeable backends.

## Status

**0.4.0 — Local Runtime and Operational Model**

Pipelantic provides the typed modeling kernel, contract interoperability,
an immutable secret-free `PipelinePlan`, and a local async runtime that
executes plans via Python callables, in-memory artifacts, and stdlib
JSON/CSV bindings.

Use Pipelantic when you want to catch incompatible wiring before processing
data, generate ODCS/DTCS/DPCS contracts from Python, or inspect a deterministic
execution plan independently of a future backend.

Pipelantic is currently alpha. Pandas, Polars, SQL, Spark, Airflow, and other
external backend plugins are design work and are not included in 0.4.

See the [hosted documentation](https://pipelantic.readthedocs.io/) for the
full design,
[CHANGELOG.md](CHANGELOG.md) for release notes, and
[Roadmap](docs/11_DEVELOPMENT/ROADMAP.md) for sequencing.

## Install

```bash
pip install pipelantic
# or
uv add pipelantic
```

### Development

Requires [uv](https://docs.astral.sh/uv/).

```bash
uv sync
uv run pytest
uv run ruff check .
uv run ruff format .
```

`uv sync` creates `.venv`, installs the package in editable mode, and installs
the `dev` dependency group (pytest, ruff, mkdocs) by default.

## Release

Tag a version that matches `src/pipelantic/_version.py`, then push the tag:

```bash
git tag v0.4.0
git push origin v0.4.0
```

GitHub Actions runs checks and publishes to PyPI using the `PYPI_API_TOKEN`
repository secret.

## Quick example

```python
from pipelantic import (
    Data,
    Input,
    Output,
    Pipeline,
    PipelineRuntime,
    Sink,
    Source,
    Transformation,
)


class RawCustomer(Data):
    customer_id: int
    first_name: str
    last_name: str


class Customer(Data):
    customer_id: int
    full_name: str


class NormalizeCustomers(Transformation):
    customers: Input[RawCustomer]
    result: Output[Customer]


class CustomerPipeline(Pipeline):
    raw: Source[RawCustomer] = Source(binding="customer_source")
    normalized = NormalizeCustomers.step(customers=raw)
    curated: Sink[Customer] = Sink(
        input=normalized.result,
        binding="customer_sink",
    )


@NormalizeCustomers.implementation("local")
def normalize(customers: list[RawCustomer]) -> list[Customer]:
    return [
        Customer(
            customer_id=row.customer_id,
            full_name=f"{row.first_name} {row.last_name}",
        )
        for row in customers
    ]


CustomerPipeline.validate(profile="development").raise_for_errors()

runtime = PipelineRuntime()
runtime.memory.seed(
    "customer_source",
    [RawCustomer(customer_id=1, first_name="Ada", last_name="Lovelace")],
)
run_report = CustomerPipeline.run(profile="development", runtime=runtime)
print(runtime.memory.get("customer_sink"))
```

Run the complete tested version at
[examples/quickstart.py](examples/quickstart.py).

## Current capability boundary

| Capability | 0.4 |
|---|---|
| Typed modeling, validation, contracts, and planning | Available |
| Local Python execution and run reports | Available |
| Memory, callable, JSON, CSV, and no-write storage | Available |
| Pandas, Polars, SQL, Spark, and Airflow plugins | Not yet available |

## Documentation

- [Documentation site](https://pipelantic.readthedocs.io/)
- [Getting Started](docs/01_GETTING_STARTED/README.md)
- [Capabilities and Limitations](docs/01_GETTING_STARTED/CAPABILITIES.md)
- [Core Concepts](docs/02_FOUNDATIONS/CORE_CONCEPTS.md)
- [Architecture](docs/02_FOUNDATIONS/ARCHITECTURE.md)
- [Roadmap](docs/11_DEVELOPMENT/ROADMAP.md)

## License

MIT
