Metadata-Version: 2.5
Name: adag
Version: 0.2.37
Summary: Another DAG: a cached, content-addressed DAG engine for Python data pipelines.
Project-URL: Homepage, https://gitlab.com/cardona-oss/adag
Project-URL: Repository, https://gitlab.com/cardona-oss/adag
Project-URL: Documentation, https://cardona-oss.gitlab.io/adag/
Project-URL: Changelog, https://gitlab.com/cardona-oss/adag/-/blob/main/CHANGELOG.md
Project-URL: Issues, https://gitlab.com/cardona-oss/adag/-/issues
Author-email: Jorge Cardona <jorgeecardona@gmail.com>
License-Expression: MIT
License-File: LICENSE
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Libraries
Classifier: Typing :: Typed
Requires-Python: >=3.12
Requires-Dist: pydantic>=2.12
Requires-Dist: structlog
Requires-Dist: typing-extensions>=4.4
Provides-Extra: docs
Requires-Dist: mkdocs-material>=9.5; extra == 'docs'
Requires-Dist: mkdocs>=1.6; extra == 'docs'
Requires-Dist: mkdocstrings[python]>=0.27; extra == 'docs'
Provides-Extra: redis
Requires-Dist: redis>=5.0; extra == 'redis'
Provides-Extra: sql
Requires-Dist: sqlalchemy>=2.0; extra == 'sql'
Description-Content-Type: text/markdown

# adag

[![pipeline status](https://gitlab.com/cardona-oss/adag/badges/main/pipeline.svg)](https://gitlab.com/cardona-oss/adag/-/pipelines)
[![PyPI version](https://img.shields.io/pypi/v/adag.svg)](https://pypi.org/project/adag/)
[![Python versions](https://img.shields.io/pypi/pyversions/adag.svg)](https://pypi.org/project/adag/)
[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](https://gitlab.com/cardona-oss/adag/-/blob/main/LICENSE)

Another DAG: a cached, content-addressed DAG engine for Python data pipelines.

## Install

```bash
pip install adag
```

Requires Python 3.12+.

## Quickstart

Each step is a frozen dataclass. Call a step to wire it into the graph, then run the
graph with a `Pipeline`.

```python
from dataclasses import dataclass
from pathlib import Path

from pydantic import BaseModel

from adag import InputParam, Pipeline, Step


class Number(BaseModel):
    value: int


@dataclass(frozen=True)
class Square(Step[Number]):
    def run(self, x: int) -> Number:
        print("computing square")
        return Number(value=x * x)


@dataclass(frozen=True)
class AddOne(Step[Number]):
    def run(self, n: Number) -> Number:
        return Number(value=n.value + 1)


x = InputParam[int]("x")
total = AddOne()(n=Square()(x=x))
pipeline = Pipeline(total, cache_dir=Path("cache"))

print(pipeline.execute(x=4))  # computes both steps
print(pipeline.execute(x=4))  # served from the cache
```

The second run reads both results from `cache/` and does not call `Square.run` again:

```text
computing square
value=17
value=17
```

(output trimmed: structlog lines removed)

## Why adag

A step's cache key is a hash of its fields, its inputs, and its source code. Change any of
them and that step and everything downstream recompute; nothing else does. Use it for data
and ML pipelines where steps are slow and you rerun the graph often. How it compares to Luigi,
Airflow, Prefect, Dagster, Hamilton and redun: [Why adag](https://cardona-oss.gitlab.io/adag/comparison/).

## Key features

- Content-addressed cache: a code edit invalidates only the steps it touches.
- `Config` fields stay out of the cache key (worker counts, clients, timeouts).
- Sync and async steps (`Step`, `AsyncStep`), with a parallel executor.
- `BatchStep` caches each item on its own, so a rerun resumes after a failure.
- `FanOutStep` runs a sub-graph per key.
- Cache stores for step results and `BatchStep` items: filesystem, memory, SQLite, SQLAlchemy
  (`adag[sql]`), Redis (`adag[redis]`). Pass one with `Pipeline(..., store=)`.
- Save a pipeline to disk and load it back with identical hashes.
- SVG rendering of the graph, and an `adag run pipeline.py` CLI.
- Run fan-out branches in worker processes or on a job scheduler (AWS Batch, SLURM, Kubernetes):
  write `submit`, `status` and `cancel`; adag does retries, timeouts and cancel.

## Links

- **Documentation:** https://cardona-oss.gitlab.io/adag/
- **Source:** https://gitlab.com/cardona-oss/adag
- **Issues:** https://gitlab.com/cardona-oss/adag/-/issues
- **PyPI:** https://pypi.org/project/adag/
- **Changelog:** [CHANGELOG.md](https://gitlab.com/cardona-oss/adag/-/blob/main/CHANGELOG.md)

## Contributing

Developer setup, the dev loop, and the release process live in
[CONTRIBUTING.md](https://gitlab.com/cardona-oss/adag/-/blob/main/CONTRIBUTING.md). In short: `uv sync && make install-hooks`, then
`make check` runs the same gate CI runs.

## License

MIT © 2026 Jorge Cardona (see [LICENSE](https://gitlab.com/cardona-oss/adag/-/blob/main/LICENSE)).
