Metadata-Version: 2.4
Name: ducklake-delta-exporter
Version: 0.1.1
Summary: A utility to export DuckLake database metadata to Delta Lake transaction logs.
Home-page: https://github.com/djouallah/ducklake_delta_exporter
Author: mim
Author-email: your.email@example.com
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Intended Audience :: Developers
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Development Status :: 3 - Alpha
Requires-Python: >=3.8
Description-Content-Type: text/markdown
Requires-Dist: duckdb
Requires-Dist: pyarrow
Dynamic: author
Dynamic: author-email
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

# 🦆 DuckLake Delta Exporter

A Python utility to **bridge the gap between DuckLake and Delta Lake** by generating Delta-compatible transaction logs directly from DuckLake metadata.

This isn’t your typical general-purpose library. It’s mostly battle-tested with **OneLake mounted storage**, and while it *should* work with local filesystems, there’s **no support for S3, GCS, or ABFSS** .

It doesn’t use the `deltalake` Python package either. The metadata is handcrafted from scratch — because why not reinvent the wheel for fun and learning?

**Goal?**  
Mostly to annoy DuckDB developers into finally shipping a proper Delta Lake metadata exporter  😎

🔗 [Source code on GitHub](https://github.com/djouallah/ducklake_delta_exporter)

---

## ✨ Features

- **DuckLake → Delta Sync**  
  Generates Delta Lake `_delta_log/*.json` transaction files and Parquet checkpoints from the latest DuckLake state.

- **Schema Mapping**  
  Converts DuckDB types to their Spark SQL equivalents so Delta can understand them without throwing a tantrum.

- **Change Detection**  
  Detects file-level additions/removals since the last export — keeps things incremental and tidy.

- **Checkpointing**  
  Automatically writes Delta checkpoints every N versions (configurable), so readers don’t have to replay the entire log from scratch.

---

## ⚙️ Installation & Usage

Install via pip:

```bash
pip install ducklake-delta-exporter
```

```
from ducklake_delta_exporter import generate_latest_delta_log

generate_latest_delta_log('/lakehouse/default/Files/meta.db')
```
