Metadata-Version: 2.1
Name: duckdb-appender
Version: 0.3.1
Summary: A DuckDB appender for Python
Author-email: Toan Phan <toanphan.dev@gmail.com>
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: duckdb>=1.0.0

# DuckDB PyAppender

A [DuckDB appender](https://duckdb.org/docs/data/overview#appender) for Python.

## Installation

## Usage

```py
import duckdb
from duckdb_appender import Appender

conn = duckdb.connect()

appender = Appender(conn, schema="main", table="users")
appender.append_row([1, "John"])
appender.append_row([2, "Doe"])
appender.close()

# Can also use it with context manager:
with Appender(conn, schema="main", table="users") as appender:
  appender.append_row([3, "foo"])
```

See more in the `/examples` folder.

## Why use this

- Much faster than simple INSERT statements.
- Versus storing data in a pandas DataFrame and load into DuckDB: while pandas loads the entire dataset into memory, this appender uses SQLite as a buffer that regularly flush itself (every 204,800 rows, similar to the official C++ appender)
  - This prevents memory overload on large data aggregation.
  - A little bit of crash protection - since it commits data every 204,800 rows, you'll preserve most of your progress if something goes wrong.

## Local development

This repo uses [uv](https://github.com/astral-sh/uv) to manage dependencies and
virtual environment. Dependencies are written in `pyproject.toml`, so other
tools supporting it can be used as well.

### Run test

This repo use [pytest](https://docs.pytest.org/en/stable/). To run tests within an uv environment, run:

```bash
uv run pytest
```
