Metadata-Version: 2.4
Name: condor-retl
Version: 0.1.1
Summary: Public API spine for reverse ETL authoring.
Project-URL: Homepage, https://condorgraph.com/
Project-URL: Documentation, https://docs.condorgraph.com/
Project-URL: Repository, https://github.com/condorgraph/condor-retl
Project-URL: Issues, https://github.com/condorgraph/condor-retl/issues
Author-email: "Dataration LLC (Condor)" <info@condorgraph.com>
Maintainer-email: Chris Morin <chris@condorgraph.com>
License-Expression: Elastic-2.0
License-File: LICENSE-Elastic-2.0.txt
License-File: LICENSE.txt
Keywords: customer-data,data-activation,data-engineering,etl,python,reverse-etl
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Information Technology
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Topic :: Database
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: <3.15,>=3.12
Requires-Dist: pyarrow<23,>=18
Requires-Dist: requests<3,>=2.32
Requires-Dist: sqlglot<31,>=30
Provides-Extra: bigquery
Requires-Dist: google-cloud-bigquery-storage<3,>=2.33; extra == 'bigquery'
Requires-Dist: google-cloud-bigquery<4,>=3.39; extra == 'bigquery'
Provides-Extra: databricks
Requires-Dist: databricks-sdk<1,>=0.59; extra == 'databricks'
Requires-Dist: databricks-sql-connector<5,>=4.1; extra == 'databricks'
Provides-Extra: duckdb
Requires-Dist: duckdb<2,>=1.4; extra == 'duckdb'
Provides-Extra: postgresql
Requires-Dist: psycopg[binary]<4,>=3.2; extra == 'postgresql'
Provides-Extra: snowflake
Requires-Dist: snowflake-connector-python<5,>=4.5; extra == 'snowflake'
Description-Content-Type: text/markdown

# condor-retl

Sync warehouse data to downstream partners from plain Python.

## Quick Start

Install the core library with a source backend and destination connector:

```sh
pip install "condor-retl[duckdb]" condor-retl-meta
```

Declare the rows to send and run the sync:

```python
import retl
from retl.backends.duckdb import DuckDBSqlBackend


db = DuckDBSqlBackend(
    database="warehouse.duckdb",
    source_schema="main",
    runtime_schema="retl",
)

newsletter_audience = retl.state(
    name="newsletter_audience",
    source=retl.source(
        name="newsletter_customers",
        mode="snapshot",
        backend=db.source_backend(),
        query="""
        select customer_id, email
        from customers
        where email is not null
        """,
    ),
    key={"customer_id": "customer_id"},
    identifiers=[{"type": "email", "value": "email"}],
    target=retl.target("newsletter_customers"),
)

meta = retl.destinations.load(
    "retl/meta",
    binding_name="meta_primary",
    credential_namespace="destinations.meta",
    config_namespace="destinations.meta",
)

result = retl.runner(
    name="newsletter_to_meta",
    runtime_store=db.runtime_store(),
).run(
    retl.sync(
        name="newsletter_to_meta",
        declaration=newsletter_audience,
        destination=meta,
        surface="custom_audiences",
    ),
    dry_run=True,
)

print(result.to_text())
```

Change `dry_run=True` to `dry_run=False` when the plan looks right.

Use a different source backend or partner connector when you need one:

```sh
pip install "condor-retl[snowflake]"
pip install "condor-retl[bigquery]"
pip install "condor-retl[databricks]"
pip install "condor-retl[postgresql]"

pip install condor-retl-klaviyo
pip install condor-retl-google-ads-data-manager
pip install condor-retl-bing-ads
pip install condor-retl-tiktok-ads
```

The distribution is `condor-retl`; the Python package is `retl`.

## Configuration

By default, RETL reads config and secrets from environment variables. The Meta
example above expects:

```sh
export DESTINATIONS__META__ACCESS_TOKEN="..."
export DESTINATIONS__META__AD_ACCOUNT_ID="act_..."
```

## What You Declare

- `source`: the SQL rows to read
- `state`: current facts, such as audience membership or profile attributes
- `event`: occurred facts, such as purchases or signups
- `sync`: one state or event declaration bound to one partner surface
- `destination`: the partner connector and account configuration

RETL runs syncs through durable `collect -> stage -> reconcile -> sync` phases,
so dry runs, retries, and operator evidence use the same declarations as
production sends.

## Partners

First-party connector packages currently cover:

| Package | Connector ref |
| --- | --- |
| `condor-retl-meta` | `retl/meta` |
| `condor-retl-klaviyo` | `retl/klaviyo` |
| `condor-retl-google-ads-data-manager` | `retl/google-ads-data-manager` |
| `condor-retl-bing-ads` | `retl/bing-ads` |
| `condor-retl-tiktok-ads` | `retl/tiktok-ads` |
