Metadata-Version: 2.4
Name: dbt-logbook
Version: 0.6.0
Summary: Run history for dbt. Every invocation recorded, nothing overwritten.
Author: Debabrata Saha
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: data engineering,dbt,observability,run history
Requires-Python: >=3.10
Requires-Dist: croniter>=6.2.3
Requires-Dist: fastapi>=0.110
Requires-Dist: mcp>=1.28.1
Requires-Dist: pyyaml>=6.0
Requires-Dist: typer>=0.12
Requires-Dist: uvicorn>=0.29
Description-Content-Type: text/markdown

# dbt-logbook

The self-hosted operations layer for production dbt Core. Keep dbt Core, add
the missing operational layer, stay self-hosted.

At its core: run history. dbt writes `run_results.json` and overwrites it on
the next run; dbt-logbook keeps every run in a local SQLite store and gives you
the views that history makes possible - run timelines, regressions, diffs,
scheduling, CI state, an agent-facing metadata API - with zero configuration
and zero changes to your dbt project.

![Per-model duration across runs, with a visible regression](docs/img/model.png)

## What you get

- **Run timeline**: every recorded invocation, status at a glance, failures inline
- **Per-model history**: duration sparkline across runs - see the regression the
  moment it starts, and the failed runs marked on the line
- **What changed between two runs**: checksum-based diff (added / removed /
  modified models), powered by dbt's own per-node checksums
- **Lineage**: clickable DAG from your manifest, tests hidden by default

## Quickstart

```
uvx dbt-logbook demo          # populated playground, no dbt project needed
```

In a real dbt project (any adapter - DuckDB, Snowflake, SQL Server, Postgres, ...):

```
cd your-dbt-project
uvx dbt-logbook ui            # instant read-only UI over the artifacts dbt already wrote
```

History accrues from the capture wrapper - change one line in your cron/CI:

```
dbt-logbook exec -- dbt build     # runs dbt untouched, records the run
                                  # exit code passes through exactly
```

Or ingest artifacts from anywhere (for example, downloaded CI artifacts):

```
dbt-logbook import path/to/artifacts --env prod
```

## Ask your agent about your runs (MCP)

The history store is exposed as an MCP server - the cross-run questions that
current-state tools structurally can't answer, because dbt overwrites its
artifacts:

```
claude mcp add dbt-logbook -- uvx dbt-logbook mcp     # from your dbt project dir
```

Then ask: *"what broke last night?"*, *"which models got slower this week?"*,
*"which tests are flaky?"*, *"what changed between the last two runs?"*,
*"what would state:modified rebuild?"*. Full tool list and REST equivalents:
[docs/api-contract.md](docs/api-contract.md).

## Run it as the platform (scheduler + alerts)

One process replaces cron + hope. Drop a `dbt-logbook.yml` in the project root:

```yaml
schedules:
  hourly:
    cron: "0 * * * *"
    command: dbt build
    retries: 2
notify:
  slack_webhook: https://hooks.slack.com/services/...   # or teams_webhook
  on: [failure, recovery]
```

```
dbt-logbook serve
```

You get: cron scheduling with retries, every run recorded, a Slack/Teams ping
on new failures and on recovery, auto-import of runs that happen outside the
scheduler (a `target/` watcher), and the UI - all one process, localhost only.

Keep it alive the boring way: `docker run --restart unless-stopped ...` or a
systemd unit with `Restart=on-failure`.

## State-based CI without artifact plumbing

The store already holds every environment's last-good manifest - serve it to CI
instead of copying `manifest.json` to S3:

```yaml
# in CI, against a reachable dbt-logbook serve --host ... --token ...
- run: |
    curl -sf -H "Authorization: Bearer $DBT_LOGBOOK_TOKEN" \
      "$LOGBOOK_URL/api/state/prod/manifest.json" -o ci-state/manifest.json
    dbt build --select state:modified --defer --state ci-state
```

Locally the same thing is one command: `dbt-logbook state --env prod --out ci-state`.
Binding beyond localhost requires a token; `/api/*` then demands
`Authorization: Bearer <token>`.

### The PR comment

After the build, `ci-report` turns the run into a reviewable verdict - changed
models, downstream blast radius, failures, and anything slower than its own
history (with cost deltas when a rate is configured):

```yaml
- name: dbt build against last-good state
  run: |
    curl -sf -H "Authorization: Bearer $DBT_LOGBOOK_TOKEN" \
      "$LOGBOOK_URL/api/state/prod/manifest.json" -o ci-state/manifest.json
    dbt build --select state:modified+ --defer --state ci-state
- name: PR report
  if: always()
  env:
    GH_TOKEN: ${{ github.token }}
  run: |
    uvx dbt-logbook ci-report --state-env prod \
      --server "$LOGBOOK_URL" --token "$DBT_LOGBOOK_TOKEN" > report.md
    gh pr comment ${{ github.event.pull_request.number }} \
      --edit-last --create-if-none --body-file report.md
```

## Spend visibility, no credentials

Add a rate to `dbt-logbook.yml` and the Health screen + `get_cost_summary`
MCP tool show estimated compute cost per model (duration x rate - honest
estimates, labeled as such). Where the adapter reports exact volume
(BigQuery's bytes billed), you get real numbers with zero warehouse
credentials - it's already in the artifacts dbt writes.

```yaml
cost:
  rate_per_hour: 3.50   # your warehouse's effective $/hour
```

## How it works

dbt-logbook reads only dbt's stable surfaces - the CLI and the artifact files
(`manifest.json`, `run_results.json`) - and never imports dbt internals. That is
why it works unchanged across dbt Core 1.7 through 2.0 (tested against golden
artifacts of 1.7, 1.8, 1.10, 1.11, and 2.0-alpha), and why it needs no dbt
installation of its own.

Every run's artifacts land in `.dbtlogbook/history.db` (SQLite; add
`.dbtlogbook/` to your project's `.gitignore`). Manifests are content-hashed and
gzipped, so the store stays small. Failed dbt runs are captured too - those are
the ones you'll want history for.

## Platform notes

- macOS and Linux. On Windows, `ui` and `import` are untested but should work
  (pure Python); `exec` is unsupported for now (POSIX signal semantics).
- The UI binds to localhost only.

## Health screen

`#/health` in the UI: duration regressions (latest vs median baseline), flaky
nodes (status flips across recent runs), and source freshness over time (from
`dbt source freshness` snapshots the watcher or wrapper picks up). Generated
`dbt docs` output is served at `/docs-site/` when present.

![Health: regressions, flaky nodes, freshness over time](docs/img/health.png)

## Roadmap

v0.5 is current. What ships next is decided by real usage, not by us guessing;
the leading candidate for v0.6 is **CI pull-request comments** - changed
models, downstream impact, failed tests, and duration regressions posted on
your PR, built on the existing diff/state APIs. Demand-gated and deferred
items (team/server mode, warehouse cost integrations, Windows exec, UI
rebuild) live in TODOS.md with their triggers.

License: Apache-2.0. Not affiliated with dbt Labs; "dbt" is a trademark of
dbt Labs, Inc.
