Metadata-Version: 2.1
Name: aurora_to_rs
Version: 0.1.0
Summary: Lean Aurora PostgreSQL to Redshift loader via copy_expert + S3 COPY
Home-page: https://github.com/ankitgoel888/aurora_to_rs
Author: Ankit Goel
Author-email: ankitgoel888@gmail.com
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE.txt
Requires-Dist: boto3
Requires-Dist: psycopg2

# aurora_to_rs

Lean Aurora PostgreSQL to Redshift loader using `psycopg2.copy_expert` + S3 + Redshift `COPY`.

~10x faster than DataFrame-based approaches for large tables (bypasses pandas entirely).

## Install

```bash
pip install aurora_to_rs
```

## Usage

```python
from aurora_to_rs import aurora_to_rs

loader = aurora_to_rs(
    region_name='ap-south-1',
    s3_bucket='my-bucket',
    redshift_c=redshift_conn,          # psycopg2 connection
    postgres_engine=pg_engine,          # SQLAlchemy engine
    iam_role_arn='arn:aws:iam::123:role/MY_ROLE'
)

# Full table refresh (TRUNCATE + COPY)
loader.upsert("SELECT * FROM master.items", "master.items", ['item_id'], clear_dest_table=True)

# Incremental upsert (staging-based)
loader.upsert("SELECT * FROM tran.orders WHERE dt>=current_date-1", "tran.orders", ['order_id'])

# Delete and insert
loader.delete_and_insert(
    "SELECT * FROM tran.shipment WHERE dt>=current_date-3",
    "tran.shipment", "dt>=current_date-3",
    min_timestamp='2026-02-19', timestamp_col='dt'
)

# Direct copy (no delete)
loader.copy_expert_upload("SELECT * FROM staging.temp", "staging.temp")
```

## Methods

| Method | Use Case |
|--------|----------|
| `copy_expert_upload` | Direct Aurora -> S3 -> Redshift COPY |
| `delete_and_insert` | DELETE by filter, then COPY new data |
| `upsert` | TRUNCATE+COPY (clear_dest_table=True) or staging-based upsert |
