Metadata-Version: 2.4
Name: ducktail
Version: 1.0.3
Summary: Tail DuckLake tables via CDC
Project-URL: Homepage, https://github.com/jghoman/ducktail
Project-URL: Repository, https://github.com/jghoman/ducktail
Project-URL: Issues, https://github.com/jghoman/ducktail/issues
Project-URL: Documentation, https://jghoman.github.io/ducktail/
Project-URL: Changelog, https://github.com/jghoman/ducktail/releases
Author: Jakob Homan
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: cdc,change-data-capture,duckdb,ducklake,streaming,tail
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Database
Classifier: Typing :: Typed
Requires-Python: >=3.12
Requires-Dist: click<9,>=8.0
Requires-Dist: duckdb<2,>=1.4
Requires-Dist: pyarrow<25,>=18.0
Requires-Dist: pyducklake>=1.0
Requires-Dist: pytz>=2024.1
Requires-Dist: rich<16,>=13.0
Description-Content-Type: text/markdown

# ducktail

[![PyPI version](https://img.shields.io/pypi/v/ducktail)](https://pypi.org/project/ducktail/)
[![CI](https://img.shields.io/github/actions/workflow/status/jghoman/ducktail/ci.yml?branch=main&label=CI)](https://github.com/jghoman/ducktail/actions)
[![codecov](https://codecov.io/gh/jghoman/ducktail/graph/badge.svg)](https://codecov.io/gh/jghoman/ducktail)
[![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](LICENSE)
[![Python](https://img.shields.io/pypi/pyversions/ducktail)](https://pypi.org/project/ducktail/)
[![Downloads](https://static.pepy.tech/badge/ducktail)](https://pepy.tech/project/ducktail)
[![Patches welcome](https://img.shields.io/badge/PRs-welcome-brightgreen)](https://github.com/jghoman/ducktail/pulls)
[![GitHub Actions Security Analysis with zizmor 🌈](https://github.com/jghoman/ducktail/actions/workflows/zizmor.yml/badge.svg)](https://github.com/jghoman/ducktail/actions/workflows/zizmor.yml)
[![Dependabot Updates](https://github.com/jghoman/ducktail/actions/workflows/dependabot/dependabot-updates/badge.svg)](https://github.com/jghoman/ducktail/actions/workflows/dependabot/dependabot-updates)

`tail -f` for [DuckLake](https://ducklake.select) tables.

![Screenshot](./images/illustration.jpg)

Watch inserts, deletes, and updates flow through a DuckLake table in real time,
powered by DuckLake's built-in CDC.

```
$ ducktail tail orders -C meta.duckdb

+ id=1 name=Alice amount=100.0
+ id=2 name=Bob amount=50.0
- id=2 name=Bob amount=50.0
Δ amount: 100.0 → 150.0
```

## Install

```bash
pip install ducktail        # or: uv add ducktail
```

Requires Python 3.12+ and DuckDB 1.4+ (ducklake extension loads automatically).

## Usage

### Basic

```bash
ducktail tail TABLE -C CONNECTION
```

```bash
# DuckDB metadata catalog
ducktail tail events -C meta.duckdb -d /data

# Postgres metadata catalog
ducktail tail events -C "postgres:dbname=ducklake host=localhost user=ducklake password=ducklake"
```

### Interactive mode

Full-screen TUI with color-coded rows (green inserts, red deletes, yellow updates):

```bash
ducktail tail events -C meta.duckdb -m interactive
```

### Filter what you see

```bash
# Only specific columns
ducktail tail events -C meta.duckdb -col status -col amount

# Server-side filter (pushed into the CDC query)
ducktail tail events -C meta.duckdb -f "amount > 100"

# Faster polling
ducktail tail events -C meta.duckdb -i 0.5
```

### Compose with Unix tools

Pager mode (the default) writes one line per change to stdout, so it plays
nicely with the usual suspects:

```bash
# Inserts only
ducktail tail orders -C meta.duckdb | grep '^\+'

# Deletes only
ducktail tail orders -C meta.duckdb | grep '^\-'

# Count changes per second
ducktail tail orders -C meta.duckdb | pv -l > /dev/null
```

## Options

```
ducktail tail [OPTIONS] TABLE_NAME
```

| Option | Short | Default | Description |
|--------|-------|---------|-------------|
| `TABLE_NAME` | | *(required)* | Table to tail |
| `--connection` | `-C` | *(required)* | Catalog connection string |
| `--catalog` | `-c` | `lake` | Catalog name |
| `--data-path` | `-d` | `.` | Path to DuckLake data files |
| `--namespace` | `-n` | `main` | Table namespace |
| `--interval` | `-i` | `1.0` | Poll interval (seconds) |
| `--columns` | `-col` | *(all)* | Columns to show (repeat for multiple) |
| `--filter` | `-f` | *(none)* | CDC filter expression |
| `--mode` | `-m` | `pager` | `pager` or `interactive` |

## Change symbols

| Symbol | Color | Meaning |
|--------|-------|---------|
| `+` | Green | Row inserted |
| `-` | Red | Row deleted |
| `Δ` | Yellow | Row updated — shows `column: old → new` for each changed field |

## Try the demo

The repo includes a Docker-based demo that stands up a Postgres metadata catalog
and a producer that streams a mix of inserts, deletes, and updates.

```bash
# Terminal 1 — infrastructure + producer
just demo-up
just demo-producer

# Terminal 2 — watch it happen
just demo-tail                # pager mode
just demo-tail-interactive    # or interactive mode

# Cleanup
just demo-down
```

## Development

Requires [flox](https://flox.dev/) and [Docker](https://www.docker.com/) (for integration tests and demo).

```bash
flox activate
just sync       # install deps
just test       # unit tests
just ci         # format + lint + typecheck + unit tests
```

| Recipe | What it does |
|--------|-------------|
| `just sync` | Install dependencies |
| `just fmt` | Format code |
| `just lint` | Lint |
| `just typecheck` | mypy strict |
| `just test` | Unit tests |
| `just test-integration` | Integration tests |
| `just test-all` | Both |
| `just ci` | fmt-check + lint + typecheck + test |

## License

Apache License 2.0
