Metadata-Version: 2.4
Name: duckless
Version: 0.3.1
Summary: Serverless DuckDB on GCP: run SQL or your own code on a right-sized VM in your project, then tear it down
License-Expression: Apache-2.0
Requires-Dist: typer>=0.15
Requires-Dist: google-cloud-batch>=0.17
Requires-Dist: google-cloud-config>=0.7
Requires-Dist: google-cloud-compute>=1.20
Requires-Dist: google-cloud-logging>=3.11
Requires-Dist: google-cloud-run>=0.16
Requires-Dist: google-cloud-storage>=2.18
Requires-Python: >=3.13
Project-URL: Homepage, https://github.com/tosun-si/duckless
Project-URL: Repository, https://github.com/tosun-si/duckless
Project-URL: Changelog, https://github.com/tosun-si/duckless/releases
Project-URL: Issues, https://github.com/tosun-si/duckless/issues
Description-Content-Type: text/markdown

<p align="center">
  <picture>
    <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/tosun-si/duckless/main/assets/brand/duckless-stacked-dark.svg">
    <img alt="DuckLess" src="https://raw.githubusercontent.com/tosun-si/duckless/main/assets/brand/duckless-stacked-light.svg" width="300">
  </picture>
</p>

<p align="center">
  <a href="https://pypi.org/project/duckless/"><img alt="PyPI" src="https://img.shields.io/pypi/v/duckless?color=F7B32B&labelColor=13233A"></a>
  <a href="https://tosun-si.github.io/duckless/"><img alt="Docs" src="https://img.shields.io/badge/docs-tosun--si.github.io%2Fduckless-F7B32B?labelColor=13233A"></a>
  <a href="https://github.com/tosun-si/duckless/blob/main/LICENSE"><img alt="License" src="https://img.shields.io/badge/license-Apache--2.0-F7B32B?labelColor=13233A"></a>
</p>

<p align="center">
  <b><a href="https://tosun-si.github.io/duckless/">Documentation</a></b> ·
  <a href="https://tosun-si.github.io/duckless/start/quickstart/">Quickstart</a> ·
  <a href="https://tosun-si.github.io/duckless/benchmarks/">Benchmarks</a> ·
  <a href="https://github.com/tosun-si/duckless/releases">Releases</a> ·
  <a href="CONTRIBUTING.md">Contributing</a>
</p>

Serverless DuckDB on GCP. Submit SQL or your own code; DuckLess runs it **in your project**
on a right-sized machine (Cloud Batch VM with local SSD for spill, or Cloud Run Jobs for small
jobs), reads and writes GCS with the job's own credentials (no HMAC keys), then tears it down.
Nothing runs between jobs. Optional [DuckLake](https://tosun-si.github.io/duckless/guides/ducklake/)
tables on a Cloud SQL catalog, and [Agent Skills](https://tosun-si.github.io/duckless/guides/agent-skills/)
for coding agents.

> Status: v0.3, young and tested on real projects. Changes are listed in the
> [GitHub Releases](https://github.com/tosun-si/duckless/releases); the
> [benchmarks](https://tosun-si.github.io/duckless/benchmarks/) have the measurements behind the design.

## Quick start

```bash
uv tool install duckless                     # or: pip install duckless

duckless init --project my-project --region europe-west1
# prints the lines to put in .envrc (DUCKLESS_PROJECT, DUCKLESS_BUCKET, DUCKLESS_SA, DUCKLESS_IMAGE)

duckless preflight --machine n2-highmem-32 --spot
duckless run job.sql --machine n2-highmem-32 --spot
duckless exec --image <your-image> --machine n2-highmem-16 -- dbt build
duckless status <job-id>
duckless logs <job-id> --follow
duckless result <job-id>
duckless destroy --project my-project       # removes what init created
```

`init` applies the Terraform module shipped with the CLI (`duckless/terraform`) through
Infrastructure Manager: no local Terraform, state kept in your project, re-run it to upgrade.
Teams managing infra as code can use the same module directly instead.

Full guide: [tosun-si.github.io/duckless](https://tosun-si.github.io/duckless/), from the
[quickstart](https://tosun-si.github.io/duckless/start/quickstart/) to the
[CLI reference](https://tosun-si.github.io/duckless/reference/cli/).

## Writing jobs

- **SQL** (`.sql`): `${VAR}` placeholders come from the job env (`DUCKLESS_BUCKET`, `--env K=V`).
- **Python** (`.py`): `from duckless_runtime import connect` gives a DuckDB connection with GCS auth,
  spill on local SSD, memory and threads sized to the VM.
- **Write to GCS in parallel**: `COPY … TO 'gs://…' (FORMAT parquet, PER_THREAD_OUTPUT, FILE_SIZE_BYTES '256MB')`
  is 7-8x faster than the single-writer default (~900 MB/s vs ~110 MB/s on 32 vCPU).
- **Vectorize Python logic**: a row-wise Python UDF runs at ~8k rows/s on one thread; use an
  Arrow UDF over numpy (`type="arrow"`) or SQL.

More in [Writing jobs](https://tosun-si.github.io/duckless/guides/writing-jobs/) and
[Machines, Spot and spill](https://tosun-si.github.io/duckless/guides/machines/).

## Agent Skills

DuckLess ships a Claude Code plugin with five skills: `setup`, `writing-jobs`, `sizing`,
`troubleshooting` and `ducklake`. They carry what the docs and the CLI cannot decide for you
(which machine, why a job failed) and the rules measured while building DuckLess.

```bash
duckless skills install          # into this project: .claude/skills + .agents/skills (--user: your home)
```

Or as a Claude Code plugin, kept up to date from this repository:

```
/plugin marketplace add tosun-si/duckless
/plugin install duckless@duckless
```

## Layout

Hexagonal, kept light: a pure core, ports, adapters, and one wiring point.

| Path | What |
| --- | --- |
| `duckless/core/` | Pure rules, no I/O: machine types and local SSD counts, job spec and planning, quotas, preflight |
| `duckless/ports.py` | What the service needs from outside: `Executor`, `ArtifactStore`, `LogReader`, `QuotaReader`, `InfraBootstrap`, `InfraDeployer` (Protocols) |
| `duckless/service.py` | Operations shared by the CLI, the SDK and later the SaaS control plane: functions taking ports as arguments |
| `duckless/adapters/` | GCP implementations: Cloud Batch, Cloud Run Jobs, GCS, Cloud Logging, Compute quotas, Infrastructure Manager |
| `duckless/wiring.py` | Binds the service functions to the adapters (lazily) |
| `duckless/cli.py` | `duckless` command, a driving adapter |
| `runtime/` | Runner image (`duckless_runtime`), published as `ghcr.io/tosun-si/duckless-runner`: DuckDB + `gcs` community extension, DuckLake + Cloud SQL Auth Proxy, tuned for the machine |
| `duckless/terraform/` | APIs, work bucket, least-privilege runner service account, Artifact Registry remote repository proxying the runner image, optional DuckLake catalog |
| `duckless/plugin/` | Claude Code plugin: the Agent Skills (`duckless/plugin/skills/`), listed by `.claude-plugin/marketplace.json` |
| `spike/` | The spike that validated the approach, kept as a record |

Dependency rule: `core` imports nothing else from DuckLess, `service` only `core` and `ports`,
and only `wiring` imports `adapters`.

## Contributing

Issues and pull requests are welcome: bug reports, real use cases, docs, code. See
[CONTRIBUTING.md](CONTRIBUTING.md) for the setup, the conventions and how changes are tested.

```bash
uv sync
uv run pytest
uv run ruff check . && uv run ruff format --check .
```

## License

DuckLess is open source under the [Apache License 2.0](LICENSE). You can use it, modify it and
ship it, commercially or not; keep the license and the notices.
