Metadata-Version: 2.4
Name: quicketl
Version: 1.2.0
Summary: QuickETL - Fast & Flexible Python ETL Framework with 20+ backend support via Ibis
Project-URL: Homepage, https://quicketl.com
Project-URL: Documentation, https://quicketl.com
Project-URL: Repository, https://github.com/ameijin/quicketl
Project-URL: Issues, https://github.com/ameijin/quicketl/issues
Author-email: Eiji <eiji@eidosoft.co>
License: MIT
License-File: LICENSE
Keywords: data-engineering,duckdb,elt,etl,ibis,pipeline,polars,quicketl
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Database
Classifier: Topic :: Scientific/Engineering
Classifier: Typing :: Typed
Requires-Python: >=3.12
Requires-Dist: fsspec>=2024.6
Requires-Dist: ibis-framework[duckdb,polars]>=9.0
Requires-Dist: pydantic>=2.10
Requires-Dist: pyyaml>=6.0
Requires-Dist: rich>=13.0
Requires-Dist: structlog>=24.0
Requires-Dist: typer>=0.12
Provides-Extra: all
Requires-Dist: adlfs>=2024.4; extra == 'all'
Requires-Dist: azure-storage-blob>=12.19; extra == 'all'
Requires-Dist: boto3>=1.34; extra == 'all'
Requires-Dist: gcsfs>=2024.6; extra == 'all'
Requires-Dist: google-cloud-storage>=2.14; extra == 'all'
Requires-Dist: hypothesis>=6.100; extra == 'all'
Requires-Dist: ibis-framework[bigquery]>=9.0; extra == 'all'
Requires-Dist: ibis-framework[clickhouse]>=9.0; extra == 'all'
Requires-Dist: ibis-framework[datafusion]>=9.0; extra == 'all'
Requires-Dist: ibis-framework[mysql]>=9.0; extra == 'all'
Requires-Dist: ibis-framework[postgres]>=9.0; extra == 'all'
Requires-Dist: ibis-framework[pyspark]>=9.0; extra == 'all'
Requires-Dist: ibis-framework[snowflake]>=9.0; extra == 'all'
Requires-Dist: ibis-framework[trino]>=9.0; extra == 'all'
Requires-Dist: mkdocs-gen-files>=0.5; extra == 'all'
Requires-Dist: mkdocs-literate-nav>=0.6; extra == 'all'
Requires-Dist: mkdocs-material>=9.5; extra == 'all'
Requires-Dist: mkdocs-section-index>=0.3; extra == 'all'
Requires-Dist: mkdocs>=1.6; extra == 'all'
Requires-Dist: mkdocstrings[python]>=0.25; extra == 'all'
Requires-Dist: mypy>=1.10; extra == 'all'
Requires-Dist: pandas>=2.0; extra == 'all'
Requires-Dist: pre-commit>=3.7; extra == 'all'
Requires-Dist: pymdown-extensions>=10.0; extra == 'all'
Requires-Dist: pytest-asyncio>=0.23; extra == 'all'
Requires-Dist: pytest-cov>=5.0; extra == 'all'
Requires-Dist: pytest>=8.0; extra == 'all'
Requires-Dist: ruff>=0.5; extra == 'all'
Requires-Dist: s3fs>=2024.6; extra == 'all'
Requires-Dist: setuptools>=60.0; extra == 'all'
Provides-Extra: aws
Requires-Dist: boto3>=1.34; extra == 'aws'
Requires-Dist: s3fs>=2024.6; extra == 'aws'
Provides-Extra: azure
Requires-Dist: adlfs>=2024.4; extra == 'azure'
Requires-Dist: azure-storage-blob>=12.19; extra == 'azure'
Provides-Extra: bigquery
Requires-Dist: ibis-framework[bigquery]>=9.0; extra == 'bigquery'
Provides-Extra: clickhouse
Requires-Dist: ibis-framework[clickhouse]>=9.0; extra == 'clickhouse'
Provides-Extra: datafusion
Requires-Dist: ibis-framework[datafusion]>=9.0; extra == 'datafusion'
Provides-Extra: dev
Requires-Dist: hypothesis>=6.100; extra == 'dev'
Requires-Dist: mypy>=1.10; extra == 'dev'
Requires-Dist: pre-commit>=3.7; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
Requires-Dist: pytest-cov>=5.0; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff>=0.5; extra == 'dev'
Provides-Extra: docs
Requires-Dist: mkdocs-gen-files>=0.5; extra == 'docs'
Requires-Dist: mkdocs-literate-nav>=0.6; extra == 'docs'
Requires-Dist: mkdocs-material>=9.5; extra == 'docs'
Requires-Dist: mkdocs-section-index>=0.3; extra == 'docs'
Requires-Dist: mkdocs>=1.6; extra == 'docs'
Requires-Dist: mkdocstrings[python]>=0.25; extra == 'docs'
Requires-Dist: pymdown-extensions>=10.0; extra == 'docs'
Provides-Extra: gcp
Requires-Dist: gcsfs>=2024.6; extra == 'gcp'
Requires-Dist: google-cloud-storage>=2.14; extra == 'gcp'
Provides-Extra: mysql
Requires-Dist: ibis-framework[mysql]>=9.0; extra == 'mysql'
Provides-Extra: pandas
Requires-Dist: pandas>=2.0; extra == 'pandas'
Provides-Extra: postgres
Requires-Dist: ibis-framework[postgres]>=9.0; extra == 'postgres'
Provides-Extra: snowflake
Requires-Dist: ibis-framework[snowflake]>=9.0; extra == 'snowflake'
Provides-Extra: spark
Requires-Dist: ibis-framework[pyspark]>=9.0; extra == 'spark'
Requires-Dist: setuptools>=60.0; extra == 'spark'
Provides-Extra: trino
Requires-Dist: ibis-framework[trino]>=9.0; extra == 'trino'
Description-Content-Type: text/markdown

# QuickETL

**Fast & Flexible Python ETL Framework with 20+ backend support via Ibis**

[![PyPI version](https://badge.fury.io/py/quicketl.svg)](https://pypi.org/project/quicketl/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

QuickETL is a configuration-driven ETL framework that provides a simple, unified API for data processing across multiple compute backends including DuckDB, Polars, Spark, and pandas.

**[Documentation](https://quicketl.com)** | **[GitHub](https://github.com/ameijin/quicketl)**

## Features

- **20+ Backends**: DuckDB, Polars, Spark, pandas, Snowflake, BigQuery, and more via Ibis
- **Configuration-driven**: Define pipelines in YAML with variable substitution
- **Quality Checks**: Built-in validation (not_null, unique, row_count, accepted_values)
- **12 Transforms**: filter, aggregate, join, derive_column, and more
- **CLI & Python API**: Use `quicketl run` or the Pipeline builder
- **Cloud Storage**: S3, GCS, Azure via fsspec

## Installation

```bash
pip install quicketl
```

See [installation docs](https://quicketl.com/getting-started/installation/) for backend-specific extras.

## Quick Start

```bash
# Create a new project
quicketl init my_project
cd my_project

# Run the sample pipeline
quicketl run pipelines/sample.yml
```

Or use Python:

```python
from quicketl import Pipeline

pipeline = Pipeline.from_yaml("pipeline.yml")
result = pipeline.run()
```

## Example Pipeline

```yaml
name: sales_etl
engine: duckdb

source:
  type: file
  path: data/sales.parquet

transforms:
  - op: filter
    predicate: amount > 0
  - op: aggregate
    group_by: [region]
    aggs:
      total: sum(amount)

sink:
  type: file
  path: output.parquet
```

## Documentation

Full documentation, tutorials, and API reference at **[quicketl.com](https://quicketl.com)**

- [Getting Started](https://quicketl.com/getting-started/)
- [Pipeline Configuration](https://quicketl.com/guides/configuration/)
- [Supported Backends](https://quicketl.com/guides/backends/)
- [CLI Reference](https://quicketl.com/reference/cli/)

## License

MIT License - see [LICENSE](LICENSE) for details.
