Metadata-Version: 2.4
Name: dagster-dataframely
Version: 0.0.1
Summary: A Dataframely plugin for Dagster.
Keywords: dagster,data,data-quality,dataframely,polars,validation
Author: Ozan Ozbeker
Author-email: Ozan Ozbeker <github@ozanozbeker.com>
License-Expression: Apache-2.0
License-File: LICENSE
Classifier: Development Status :: 1 - Planning
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Database
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Typing :: Typed
Requires-Dist: dagster>=1.13.16
Requires-Dist: dataframely>=3.0.0
Requires-Dist: polars>=1.43.2
Requires-Dist: pydantic>=2
Requires-Dist: universal-pathlib>=0.2.0
Maintainer: Ozan Ozbeker
Maintainer-email: Ozan Ozbeker <github@ozanozbeker.com>
Requires-Python: >=3.12
Project-URL: Homepage, https://github.com/ozanozbeker/dagster-dataframely
Project-URL: Repository, https://github.com/ozanozbeker/dagster-dataframely
Project-URL: Issues, https://github.com/ozanozbeker/dagster-dataframely/issues
Project-URL: Changelog, https://github.com/ozanozbeker/dagster-dataframely/releases
Description-Content-Type: text/markdown

# dagster-dataframely

Dataframely plugin for Dagster.

> [!WARNING]
> **Pre-release placeholder.**
> This package is under active design and ships no functionality yet.
> The name is reserved on PyPI while the design spec is finalized.
> Do not depend on it.
> Follow [issue #1](https://github.com/ozanozbeker/dagster-dataframely/issues/1) for progress.

## What it will do

[dataframely](https://github.com/Quantco/dataframely) declares schemas and validation rules for [polars](https://pola.rs) data frames.
[Dagster](https://dagster.io) has first-class surfaces for data contracts: column schema metadata and asset checks.
This package connects the two:

- **Asset checks derived from your schema.**
  Each dataframely rule becomes a Dagster asset check, so per-rule pass/fail history shows up in the catalog without hand-writing checks.
- **Column schema metadata.**
  Your schema populates Dagster's Columns tab automatically.
- **Quarantine, opt-in.**
  Declare a second output and failing rows are routed there with per-rule attribution instead of failing the run.
- **Storage in the box.**
  `DataframelyParquetIOManager` writes `.parquet` to a local directory or to `s3://`, `gs://` and `az://`, and it is the supported path.

Dependencies are `dagster` and `dataframely` (plus `polars`) only.
The IO manager imports `universal-pathlib` and `pydantic` directly, so both are declared too.
Both already ship with `dagster`, so nothing new lands in your environment.

## Installation

```bash
uv add dagster-dataframely
```

Requires Python 3.12+.

## License

[Apache-2.0](LICENSE)
