Metadata-Version: 2.5
Name: sql-code-graph
Version: 1.82.2
Summary: SQL code graph analyzer and lineage tracer
Project-URL: Homepage, https://github.com/Warhorze/sql-code-graph
Project-URL: Repository, https://github.com/Warhorze/sql-code-graph
Project-URL: Issues, https://github.com/Warhorze/sql-code-graph/issues
Project-URL: Changelog, https://github.com/Warhorze/sql-code-graph/blob/master/CHANGELOG.md
Author-email: wesley <rademakerwesley@gmail.com>
License: MIT
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Database
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.12
Requires-Dist: duckdb<2.0,>=1.0.0
Requires-Dist: mcp<2.0,>=1.27.0
Requires-Dist: pathspec>=0.12.1
Requires-Dist: pydantic>=2.0
Requires-Dist: python-dotenv>=1.0.0
Requires-Dist: rich>=13.7.0
Requires-Dist: sqlglot==30.6.0
Requires-Dist: sqlglotc==30.6.0
Requires-Dist: typer>=0.9.0
Requires-Dist: watchdog>=3.0.0
Provides-Extra: dbt
Requires-Dist: dbt-core>=1.7; extra == 'dbt'
Provides-Extra: explorer
Provides-Extra: snowflake
Requires-Dist: cryptography>=42; extra == 'snowflake'
Requires-Dist: snowflake-connector-python>=3.12; extra == 'snowflake'
Provides-Extra: tableau
Requires-Dist: tableauserverclient>=0.41; extra == 'tableau'
Description-Content-Type: text/markdown

<img src="https://raw.githubusercontent.com/Warhorze/sql-code-graph/master/docs/assets/sqlcg-logo.svg" alt="" width="64" height="64" align="left" />

# sql-code-graph

> **No backward-compatibility guarantees.** APIs, CLI flags, and graph schema may
> change between releases without a deprecation period. Pin to an exact version
> in production. Re-indexing is always the migration path.

> **Dialect support.** sqlcg is built on [sqlglot](https://github.com/tobymao/sqlglot),
> so in theory it can work for any dialect sqlglot parses. In practice we've only
> tested it against a production **Snowflake** warehouse (~1,400 SQL files) — and
> getting that one dialect working properly was already challenging. Other dialects
> (`bigquery`, `postgres`, `ansi`, `tsql`, `dbt`) may well work, but their lineage
> hasn't been validated, so expect rough edges. We'd gladly collaborate to get
> another dialect integrated — [open an issue](https://github.com/Warhorze/sql-code-graph/issues)
> with a minimal, anonymised corpus we can test against.

SQL lineage and dependency analysis as an MCP server for Claude Code.

Indexes a directory of `.sql` files into a graph database and exposes lineage
queries as MCP tools — so Claude can answer questions like *"what tables does
this view depend on?"* or *"where is `orders.customer_id` derived from?"*
without reading every file.

## Quick start

Choose one:

**Permanent install** (recommended):
```bash
uv tool install sql-code-graph    # Fast, managed, no isolation needed
sqlcg install                     # Register MCP server in Claude Code
```

**One-shot try** (cold cache warning):
```bash
uvx sql-code-graph                # First run is slow (downloads deps)
                                  # Subsequent runs use cache, ~1s startup
```

Restart Claude Code, then inside your project ask:

```
Index my SQL files at ./sql --dialect snowflake
```

That's it. The MCP tools are now available to Claude in every conversation
for that project.

### Workflow (3 steps)

1. **Initialize**: `sqlcg db init`
2. **Index**: `sqlcg index ./sql --dialect snowflake`
3. **Keep fresh**: `sqlcg git install-hooks` (auto-reindex on branch switch), or
   `sqlcg watch ./sql` to re-index automatically as you edit files (both optional)

## Full setup (recommended)

```bash
# 1. Install (this project uses uv — never pip)
uv tool install sql-code-graph          # global tool (recommended)
# — or, from a source checkout:
#   git clone https://github.com/Warhorze/sql-code-graph
#   cd sql-code-graph && uv sync        # then run via `uv run sqlcg …`

# 2. Register with Claude Code (writes ~/.claude.json)
sqlcg install

# 3. Restart Claude Code

# 4. Index your SQL repo
# Only git-tracked files are indexed — build artefacts, node_modules,
# and .venv are ignored automatically.
sqlcg db init
sqlcg index ./sql --dialect snowflake   # snowflake is the only tested dialect

# 5. (Optional) Keep the graph fresh on branch switches
cd /your/sql/repo
sqlcg git install-hooks
```

Step 5 installs a `post-checkout` git hook that re-indexes automatically
whenever you switch branches. Without it the graph may be stale after a
`git checkout` until you re-run `sqlcg index` manually.

## Dialect config

To avoid passing `--dialect` every time, create `.sqlcg.toml` in your repo root:

```toml
[sqlcg]
dialect = "snowflake"   # the only tested dialect
```

The git hook and `sqlcg index --dialect auto` both read this file.

## Add to your project CLAUDE.md (recommended)

Adding a short note to your project's `CLAUDE.md` helps Claude know the tools
are available and when to use them:

```markdown
## SQL lineage
This project uses sql-code-graph. MCP tools are available:
- `db_info` — check graph health and parse quality before running lineage queries
- `index_repo` — index or re-index a directory of SQL files
- `find_table_usages` — find all queries that read a table
- `trace_column_lineage` — trace where a column's value comes from
- `get_upstream_dependencies` / `get_downstream_dependencies` — dependency chains
- `search_sql_pattern` — full-text search across all indexed SQL
- `execute_sql` — raw read-only SQL query for advanced analysis
```

The MCP server works without this — Claude can discover the tools on its own —
but the CLAUDE.md snippet ensures they get used proactively.

## Parse quality

After indexing, `sqlcg gain` shows a **parse quality breakdown** that tells you how
much column-level lineage was extracted:

| Quality | Meaning | Tools affected |
|---|---|---|
| `FULL` | Column-level lineage extracted | All tools work |
| `TABLE_ONLY` | Table edges only — no column lineage | `trace_column_lineage`, `get_*_dependencies` return empty |
| `SCRIPTING_FALLBACK` | sqlglot fell back to raw command node | Partial table edges; column lineage unavailable |
| `FAILED` | File failed to parse entirely | File invisible to all queries |

Quality is shown per-file after `sqlcg index` and in `sqlcg gain` Section F.
`list_dialects_and_repos()` warns when scripting fallback exceeds 20% of queries.

For every other `gain` metric — edge health, phantom edges, blindspot tables,
ground-truth precision/recall (table- and column-level) — see
[docs/metrics.md](docs/metrics.md), the authoritative glossary of formula,
numerator, denominator, and in/out membership for each line.

**What causes TABLE_ONLY?** Mostly `SELECT *` — sqlglot can't trace column names through
a wildcard without knowing the source table's columns. See [Resolving SELECT *](#resolving-select-)
below for how sqlcg expands wildcards automatically from your DDL and CTAS bodies.

**What causes SCRIPTING_FALLBACK?** Snowflake `$$` procedure bodies or `BEGIN…END` scripting
blocks. sqlglot parses the block as a raw `Command` node and extracts DML via tokenizer
fallback. Table edges are usually correct; column edges are not.

Check `sqlcg db info` for the parsing mode distribution across all indexed queries.

## Resolving SELECT *

A `SELECT *` ETL produces `TABLE_ONLY` parse quality because sqlglot needs the source
table's column list to expand the wildcard into individual columns. sqlcg resolves this
**automatically, with no extra setup** — there is no CSV to export or command to run.

Wildcards are expanded from two sources harvested while indexing:

1. **DDL files** — `CREATE TABLE` / `CREATE VIEW` statements give sqlcg the column list
   for any table they define.
2. **Cross-file CTAS bodies** — a `CREATE TABLE … AS SELECT` (or CTE) in one file is used
   to resolve `SELECT *` against that table in another file.

So the only thing you need to do is **index the DDL alongside your ETLs** — point `sqlcg
index` at a path that contains both (e.g. the repo root, or both `ddl/` and `etl/`). The
more of your `CREATE` statements sqlcg can see, the more wildcards it resolves.

```bash
sqlcg index . --dialect snowflake   # index DDL + ETLs together
```

After indexing, `sqlcg db info` shows non-zero `STAR_EXPANSION lineage edges`, and
`trace_column_lineage` returns results for queries that previously returned empty.

> **Note:** an earlier `sqlcg load-schema` command was removed in 2026-05 — it fed
> column names into the parse/qualify loop (hot path) and measured zero edge delta on
> a real warehouse.  The new `sqlcg catalog load` (see below) is different: it writes
> `HAS_COLUMN` rows **after** indexing and never touches the parser.

## INFORMATION_SCHEMA catalog enrichment

If your warehouse DDL lives in XML changelogs (Liquibase, Flyway) or other non-SQL
formats that sqlcg cannot parse, you can supply an `INFORMATION_SCHEMA.COLUMNS` export
to enrich edge health without re-parsing anything.

```bash
# Export from Snowflake (or any SQL client):
# SELECT table_catalog, table_schema, table_name, column_name, ordinal_position, data_type
# FROM information_schema.columns
# WHERE table_schema NOT IN ('INFORMATION_SCHEMA')
# ORDER BY table_schema, table_name, ordinal_position;
# → save as columns.csv

sqlcg catalog load columns.csv
sqlcg gain          # check the delta
```

**What it does:**
- Reads `table_catalog` (optional), `table_schema`, `table_name`, `column_name` columns.
- Delimiter sniffed (`,` or `;`); extra columns ignored; names case-folded to lowercase.
- Creates `SqlColumn` nodes and `HAS_COLUMN(source='information_schema')` edges.
- DDL-sourced rows are never overwritten (`ddl > information_schema > usage` precedence).
- Idempotent — running it twice produces no row growth.
- Catalog-sourced rows survive per-file reindex (`sqlcg reindex`).

**Persisting across full rebuilds:** add to `.sqlcg.toml`:

```toml
[sqlcg.catalog]
path = "/path/to/columns.csv"
```

With this set, `sqlcg index` automatically re-applies the catalog at the end of each
full rebuild. If the file is missing, indexing succeeds with a warning.

**What it does not do:**
- It does not feed column names into the parse/qualify hot path — `SELECT *` expansion
  uses only DDL files and cross-file CTAS bodies (unchanged).
- It does not create new lineage edges — it only validates / invalidates existing ones.

## Observed-usage store (`usage load` + `gain --from-store`)

`sqlcg gain` measures lineage precision/recall against **observed-usage** ground truth —
what the warehouse actually did (from `ACCESS_HISTORY` / `OBJECT_DEPENDENCIES`). That
ground-truth data can come from three interchangeable sources, all landing in one typed
local sidecar — `usage.db`, next to `graph.db` — and read back the same way regardless of
which source filled it. `usage.db` is created **only** when you run `usage load`; a plain
`sqlcg index` never creates or touches it, so there is zero setup for anyone who does not
use this.

### Which flavor is for me?

| Your situation | Flavor | How |
|----------------|--------|-----|
| No Snowflake access — you have (or can export) a CSV by hand | **1 — manual CSV** | `usage load` (below) |
| You want a kept-fresh local copy that *we* pull on a schedule (unattended) | **2 — daily incremental sync** | `sqlcg usage sync` (future) |
| The customer maintains the table with their own jobs; you just read it live | **3 — live customer table** | `sqlcg usage link` (future) |

Decide in one read: **no Snowflake access → flavor 1**; **want a scheduled sync we run →
flavor 2**; **customer owns the table and we read it live → flavor 3.** Flavors 2 and 3 are
opt-in future surfaces (sketched at the end); flavor 1 ships today and is fully offline.

### Per-flavor requirements — AUTH is the thing to get right

| Flavor | Snowflake access | **Auth** | Setup |
|--------|------------------|----------|-------|
| **1 — manual CSV** | None | **None** | Zero — just a CSV file. |
| **2 — daily incremental (scheduled / unattended)** | Read on the source tables | **Non-interactive only: PAT (recommended) or key-pair (RSA). NOT interactive SSO.** | A named connection in `~/.snowflake/connections.toml` set up for PAT or key-pair. |
| **3 — live customer table** | Read on the customer table | PAT / key-pair for any automation; interactive SSO acceptable only for hands-on use, never a scheduled job. | A named connection + the customer table reference in `.sqlcg.toml`. |

**Why not SSO for anything unattended:** `externalbrowser` (interactive SSO) **hangs in
headless / scheduled contexts** — it needs a human to complete a browser flow, which never
happens under cron or CI. This is a hard, learned constraint, so any scheduled / unattended
pull (flavor 2, and any automated flavor 3) **must** use PAT or key-pair. Interactive SSO is
fine only for a human running a command by hand.

### Setting up PAT auth (recommended for headless) — zero sqlcg code

sqlcg's Snowflake connection is **auth-agnostic**: it delegates entirely to a *named
connection* in `~/.snowflake/connections.toml`, which owns the account, user, role,
warehouse, database, **and** the authenticator. Switching to PAT (or key-pair) is pure
config — **no sqlcg code changes**. The connector supports PAT natively via
`authenticator = "PROGRAMMATIC_ACCESS_TOKEN"`.

Add a named connection (every value here is a **placeholder** — fill in your own):

```toml
# ~/.snowflake/connections.toml
[<connection-name>]
account = "<account-identifier>"
user = "<user>"
authenticator = "PROGRAMMATIC_ACCESS_TOKEN"
token = "<programmatic-access-token>"
role = "<role>"
warehouse = "<warehouse>"
database = "<database>"
```

Then lock the file down — the connector **refuses** a group- or world-readable connections
file:

```bash
chmod 0600 ~/.snowflake/connections.toml
```

**PAT caveat:** programmatic access tokens are role-scoped and **expire**, so they must be
rotated periodically. If you want a headless credential that does **not** expire, use
**key-pair** auth instead — `authenticator = "SNOWFLAKE_JWT"` with
`private_key_file = "<path-to-private-key>"` in the same named-connection block. Key-pair is
the non-expiring alternative and needs no sqlcg code either.

### Telling sqlcg which connection to use

sqlcg owns exactly one piece of Snowflake config — *which named connection*. Set it per
project in `.sqlcg.toml`:

```toml
[sqlcg.snowflake]
connection_name = "<connection-name>"
```

…or pass `--connection-name` on the command line (for example,
`sqlcg gain --from-snowflake --connection-name <connection-name>`).

> **Do not** set the global `default_connection_name` in `~/.snowflake/config.toml` for
> this. That key is **global** — it would hijack interactive `snowsql` and every other tool
> that resolves the default connection. Prefer the per-project `[sqlcg.snowflake]
> connection_name` above, or the `--connection-name` flag.

### Flavor 1 — load a CSV (available today)

```bash
sqlcg usage load oracle_edges.csv --dataset oracle-edges
sqlcg gain --from-store                 # measure precision/recall offline, no Snowflake
```

`usage load` casts every column to its declared type on load (typed DuckDB `read_csv`); a
value that cannot cast **raises** rather than silently becoming a string. Re-running for the
same dataset fully replaces its table (no row growth). The registered datasets:

| `--dataset` | `load_mode` | Holds |
|-------------|-------------|-------|
| `oracle-edges` | `incremental` | baseSources-derived write edges (src/dst object + BI/DDL/ephemeral flags) |
| `direct-sources` | `incremental` | `directSources` guard edges (src/dst object) |
| `object-dependencies` | `incremental` | `OBJECT_DEPENDENCIES` view-lineage guard edges |
| `catalog` | `replace` | `INFORMATION_SCHEMA.COLUMNS` rows — backs `catalog load` from the store |

### `load_mode`, staleness, and watermarks

Each dataset carries a **`load_mode`**:

- **`replace`** — every load is a full overwrite. Correct for data with no event time (the
  `catalog` dataset: "the current catalog" is the only correct state). Never
  staleness-checked.
- **`incremental`** — freshness matters; a scheduled sync appends new rows over a lag
  re-pull window, and the dataset's age is tracked. Applies to the oracle ground-truth
  datasets.

**Staleness / TTL.** Incremental datasets are gated by the **same** config key the live
ground-truth cache already uses — no new key:

```toml
[sqlcg.snowflake]
ground_truth_ttl_hours = 24    # default 24h; incremental datasets older than this are stale
```

`replace` datasets are never stale (there is no event time to age).

**Watermarks (for incremental sync).** The correct, **recommended** way to track
incremental freshness is a real **event-time column** on your data (a `loaded_at`,
`event_ts`, etc.) declared as the dataset's `watermark_column`. Only a real event-time
column can catch **late arrivals** — a row whose event time falls inside the lag window but
which was loaded after the last sync. When no event-time column exists, the store falls back
to self-stamping a `last_updated_date` at load time; but that is a *load* stamp, not an
*event* stamp — it advances on every load and therefore **cannot catch late arrivals**.
Declare an event-time `watermark_column` whenever you have one.

### Flavors 2 & 3 (future, opt-in — sketch)

- **`sqlcg usage sync`** (flavor 2) — a scheduled / unattended pull that opens a Snowflake
  connection, reads the current watermark, queries the lag window, and appends only new,
  deduped rows to the same typed table. Must use PAT / key-pair (see the auth table above).
- **`sqlcg usage link`** (flavor 3) — points sqlcg at a live customer-owned table and
  materializes a typed *view* over it (no local copy); reads reflect the customer's latest
  data on every run.

Both are opt-in and sketched here only so the surface is named; the auth requirements above
already apply to them because they are learned constraints, not speculation.

## Excluding files & re-including for full coverage

`sqlcg` honors a `.sqlcgignore` file (gitwildmatch globs, same syntax as `.gitignore`)
at the index root. Any path matched by a rule there is skipped at **file discovery**
— it is never parsed, so it contributes zero lineage.

This matters if a directory was excluded as a perf workaround. For example, a rule
like:

```
sql/generated/wide_dynamic_tables/
```

might exclude a directory of wide `CREATE OR REPLACE DYNAMIC TABLE` (or other
wide-body) definitions. **Removing that line from `.sqlcgignore` re-includes the
directory and recovers its lineage** — this is a change to *your* config, not to
`sqlcg` itself; no flag or code change is involved.

**Measured recovery (2026-07-05, on a ~1,555-file warehouse corpus):** re-including
a ~116-file wide dynamic-table directory recovered **≈+4,790 `COLUMN_LINEAGE`
edges**, and all 114 of the 114 wide files that could carry lineage gained it (from
zero). Keep `[sqlcg.catalog] path = ...` (above) configured — the post-index catalog
re-apply is required for these files to resolve fully.

**Memory caveat — read before re-including a large excluded directory:** re-including
a wide directory needs headroom; a full warehouse-scale index peaks around **~1.5 GB
physical RSS**, and that peak is a **worker-count-independent main-process floor**
(the batched-upsert working set plus the catalog re-apply), not something driven by
file width. The `--n-workers` flag lets you lower worker concurrency, but it does
**not** lower this physical peak on this workload — it is not a substitute for having
enough free RAM. On a box with less than ~1.5 GB free, re-inclusion may still OOM
regardless of `--n-workers`; that constrained-box case is a known limitation, not
something `sqlcg` can currently work around.

Wall time is not the constraint: a full re-inclusion of the example above took a few
minutes (measured ~4-5 min end-to-end on the ~1,555-file corpus) at default settings.
If you have the RAM, re-inclusion is affordable to run.

## MCP tools reference

| Tool | Description |
|------|-------------|
| `index_repo(repo_path, dialect)` | Index a directory of SQL files |
| **Lineage & dependencies** | |
| `trace_column_lineage(table_col)` | Trace a column's value upstream to its sources |
| `get_upstream_dependencies(table_col)` | Full upstream dependency chain |
| `get_downstream_dependencies(table_col)` | Full downstream dependency chain |
| `find_table_usages(table_name)` | Find all queries that read a table (3-signal: direct SELECT, SELECT *, or column-lineage CTE read) |
| `find_definition(table_qualified)` | Find where a table/view is defined |
| **Change impact** | |
| `get_change_scope(table_qualified)` | Blast radius of changing a table (impact + risk); `gating_join_tables` field adds row-reachability supplement (see caveats) |
| `diff_impact(changed_files)` | What a set of changed files affects downstream; `gating_join_tables` field adds row-reachability supplement (see caveats) |
| `get_backfill_order(table_qualified)` | Topological rebuild/backfill order |
| `scope_change(target)` | Synthesised change-scope summary for a target |
| `get_empty_propagation(tables)` | Downstream blast radius when named table(s) are empty — two views: value-derivation (primary) + row-reachability (supplement) |
| `get_pr_impact(base_ref)` | Detect producers a PR dropped + their blast radius; code-regression detection, not runtime monitoring |
| **Analysis** | |
| `get_hub_ranking(k)` | Top-k tables by downstream dependent count (hub/centrality) |
| `analyze_unused()` | Tables with no within-corpus consumers (dead-code candidates, heuristic); uses 3-signal "used" definition — direct SELECT, SELECT *, or CTE-derived column read |
| **Search & meta** | |
| `search_sql_pattern(query)` | Full-text search across indexed SQL |
| `list_dialects_and_repos()` | List indexed repos and dialects (catalogue) |
| `db_info()` | Graph health, node counts, parse quality breakdown, warnings, freshness (indexed SHA vs HEAD) |
| `execute_sql(query)` | Raw read-only SQL query against the graph |
| `submit_feedback(...)` | Report a false positive/negative to improve metrics |
| **Lineage explorer contract (O1-O7, O9)** | Thin adapters over `sqlcg.api.operations` — every response is the same JSON envelope the `sqlcg explorer serve` HTTP transport and `--json` CLI flags return (see `contract/schema/*.json`) |
| `search_objects(query, kinds, limit, cursor, raw)` | O1 — search tables, columns AND BI (Tableau/etc.) consumers by substring |
| `get_node(key)` | O2 — detail card for one table/column/consumer key |
| `expand_neighbors(anchor, direction, grain, depth, limit, cursor, raw, include_intermediate)` | O3 — multi-hop lineage expansion, both grains, both directions, paged |
| `get_edge_detail(src, dst, edge_type)` | O4 — producing SQL body (file:line + text) behind one lineage edge |
| `list_table_columns(table_key, limit, cursor, order)` | O5 — paged column listing for one table (handles 500+ column tables) |
| `get_graph_health()` | O6 — coverage/lineage-completeness/BI-freshness counters for the whole graph |
| `get_empty_impact(table_key, limit, cursor)` | O7 — ranked blast-radius of one table going empty, as two never-merged groups |
| `list_file_objects(path)` | O9 — which tables/views one SQL file defines or populates, with line ranges. Path matched against the index as a string (verbatim, repo-root-stripped, repo-root-prefixed); a file the index has never seen returns `objects: []` with `NOT_INDEXED`, not an error |

> **Input format**: lineage/dependency tools expect a **schema-qualified** column
> reference — `schema.table.column` (e.g. `ba.orders.customer_id`), not a bare
> `table.column`. Each returned node carries both `name` (the bare column) and
> `table` (the owning `schema.table`), so results are navigable without a second lookup.

> **Provenance fields**: lineage edges now carry `file`, `line`, and `expression`
> (where the lineage was derived from), a `confidence` of `1.0` for plainly-parsed
> facts (lower for inferred edges, with a `reason`), and a `table_kind`
> (`table` / `cte` / `derived` / `external`) so CTE and derived aliases are
> distinguishable from real tables.

> **LLM agent tip**: call `db_info()` before lineage queries to check that
> `SqlColumn > 0` and `warnings` is empty. If `parse_quality["scripting_block"]`
> is high, column lineage will be limited for those files — use table-level tools
> (`find_table_usages`, `get_*_dependencies`) instead.

## CLI reference

Full option reference: [docs/cli.md](docs/cli.md)

```bash
sqlcg install                          # register MCP server in Claude Code
sqlcg db init                          # initialise graph database
sqlcg index <path> --dialect snowflake # index SQL files (snowflake is the tested dialect)
sqlcg index <path> --dialect auto      # read dialect from .sqlcg.toml
sqlcg index <path> --profile           # index + print per-stage timing and slowest files
sqlcg index <path> --include-working-tree  # also index uncommitted changes (marks graph dirty)
sqlcg reindex <path> --from <sha> --to <sha>  # incremental resync of only changed files
sqlcg analyze unused                   # tables with no within-corpus consumers (3-signal: direct SELECT, SELECT *, or CTE-derived column read)
sqlcg analyze empty-impact <table>...  # downstream blast radius when named table(s) are empty (two views)
sqlcg analyze pr-impact --base <ref>   # detect producers a PR dropped + their blast radius (code-regression detection)
sqlcg analyze upstream/downstream      # trace lineage from the CLI
sqlcg find table/column/pattern        # search the graph
sqlcg analyze upstream/downstream/impact/empty-impact --json  # same contract envelope as `explorer serve` (contract/schema/*.json)
sqlcg find table/column --json         # same contract envelope as `explorer serve` (contract/schema/search_objects.json)
sqlcg watch <path>                     # watch a directory and re-index on SQL file changes
sqlcg viz                              # generate a self-contained graph-explorer HTML
sqlcg explorer serve                   # serve the interactive lineage explorer UI + API on http://127.0.0.1:8765
sqlcg db info                          # graph stats + freshness (indexed SHA vs HEAD)
sqlcg git install-hooks                # install post-checkout + post-merge resync hooks
sqlcg gain                             # show usage metrics
sqlcg report                           # generate FP/error report
sqlcg mcp best-practices               # print the fact/heuristic boundary for the MCP tools
sqlcg mcp start                        # start MCP server manually
sqlcg mcp status                       # server status JSON — incl. running version + stale_by_version
sqlcg mcp stop                         # stop the running MCP server gracefully
sqlcg mcp restart                      # stop the server (client must respawn it)
sqlcg version                          # show installed version
```

### Staying on the latest build

The installed package, the CLI, and the running MCP server all report the **same**
version (`sqlcg.__version__`). After upgrading, an editor may still be talking to an
old MCP process — `sqlcg mcp status` surfaces this directly: it reports the running
server's `version` and a `stale_by_version` flag that is `true` when the live server
differs from the installed build. Re-running `sqlcg install` stops the stale server so
your editor respawns it on the new build, so you never debug against an outdated server.

### Reads while the server is running

DuckDB takes an exclusive lock on the database file, so while the MCP server is
live it holds that lock (other processes cannot open the file, even read-only).
CLI **read** commands (`find`, `analyze`, `db info`, `list-repos`,
`gain`) automatically route their query through the running server over its
control socket and return rows as usual — no flag, no config. When no server is
running they open the database directly, exactly as before. If the server is
mid-reindex the read waits for it to finish rather than failing with
"Database is locked".

### Visualising the graph (`viz`)

`sqlcg viz` generates a **self-contained graph-explorer HTML** from the live
graph — every node, edge, the force-graph library, and the facet data are inlined
into a single file, so it opens by double-click in a browser with no server or
external resources.

```bash
sqlcg viz                              # writes table_graph.html in the current dir
sqlcg viz --out lineage.html           # choose the output path
```

In the browser the filter composes schema ∩ kind ∩ tag (multi-select per facet)
plus a job dropdown; an edge renders only when both endpoints are visible. Table,
view, and temp nodes are on by default; CTE and derived nodes are off (toggleable).

Two optional CSV facets let you colour and slice the graph by your own
conventions — both are `pattern,label[,color]` with a required header, patterns
are case-insensitive `fnmatch` globs (`*`, `?`) matched against the qualified
table name:

```bash
sqlcg viz --tags tags.csv              # legend swatches: colour + filter the graph
sqlcg viz --jobs jobs.csv              # populate the job dropdown (filter only)
```

Omit `--tags` to hide the tag legend; omit `--jobs` to leave the dropdown empty.

### The lineage explorer (`explorer serve`)

`sqlcg explorer serve` starts a **read-only, loopback-only** HTTP server on
`http://127.0.0.1:8765` (`--port` to change it) and serves an interactive lineage
explorer UI backed by the live graph. There is no `--host`
flag: it only ever binds to `127.0.0.1`, has no auth, and exposes no write path
(no `index`/`reindex`/`bi ingest` over HTTP) — see
[`ARCHITECTURE_REVIEW.md`](ARCHITECTURE_REVIEW.md) §9.2 for the full constraint
list. Safe to run alongside a live `sqlcg mcp` server: reads are routed the same
way CLI reads already are, so the explorer never opens the database file a
second time.

```bash
sqlcg explorer serve                   # http://127.0.0.1:8765, uses $SQLCG_DB_PATH / ~/.sqlcg/graph.db
sqlcg explorer serve --port 9000       # choose a different loopback port
sqlcg explorer serve --db-path /path/to/graph.db  # point at a specific graph
```

**The explorer can follow the file you have open.** `editors/vscode/` is a VS Code
extension that puts the explorer in a side panel and points it at the object your
cursor is in, as you move between statements and files. It is built and installed
from a locally packaged `.vsix` (not from the marketplace); see
[`editors/vscode/README.md`](editors/vscode/README.md) for the build, the three
settings and a manual verification script. VS Code is the only editor client —
Zed's extension API has no panel or webview surface as of 2026-09.

Two optional `[sqlcg.toml]` keys add facets to the map — the same BYO CSVs
`sqlcg viz` takes via `--tags` and `--jobs`, read from config because `serve` is long-running:

```toml
[sqlcg.viz]
tags = "viz/tags.csv"   # pattern,label[,color] -> `tag` chips + a ring on tagged tables
jobs = "viz/jobs.csv"   # pattern,label         -> the `job` dropdown (a filter)
```

Paths resolve relative to the directory holding `.sqlcg.toml`. With neither key set
the bar shows no tag or job control at all; a configured-but-unreadable CSV says so in
one line instead of failing silently.

**The home screen is the whole graph, not a search box.** The map loads once, in one
read, and draws every table, view and dashboard the index knows about; the facet bar
above it (search, `schema`, `kind`, `tag`, `job`, `min connections`, `highlight depth`)
filters what is drawn, and clicking a node opens its details beside the map — what feeds
it, what it feeds, which dashboards read it, its columns and their lineage, and its
blast radius on demand. There is no anchor to type first and no round trip per hop:
every operation is an action on the current selection.

**Every link on the map says how sure the tool is about it, and every row can say why.**
Certainty is drawn on one channel — the line style: solid is *certain* (the tool read the
link out of the statement), dashed *likely*, dotted *best effort*, thin dashed *unchecked*.
The `certainty floor` chips fade everything below the level you pick and report what they
faded (`Certainty floor 'Likely' — 1,204 of 5,836 links faded, none removed`); nothing is
removed from the map and no count anywhere changes, so raising the floor can never hide a
table from you.

Under `Fed by`, `Feeds`, `Dashboards` and `Column lineage`, every row that is one step away
carries a `Why?` button. It opens that one link's story in place of the panel: which
statement produced it, how sure the tool is and why in one sentence, the file and line it
was read from, and the statement itself with the producing line highlighted. Raise the
highlight depth and the rows further out arrive without the button — there is no single
link to explain two hops away — and the section says so once, in one line. A dashboard link has no SQL to show and says so,
with the date the BI tool recorded it. Nothing on any of those screens is an internal token
or a confidence score — named states and plain sentences only.

**A selected table can highlight just one side, and you can mark where the lineage starts.**
The `Map options` popover's `show` chips — `both` (default), `fed by`, `feeds` — narrow which
direction of the selection the map highlights, using the same amber/cyan the legend and the
panel already teach; they never change how far the highlight reaches (that stays `highlight
depth`). Under `fed by` or `feeds`, a `hide the other side` checkbox appears and, once ticked,
removes the other direction from the map instead of dimming it (the map re-frames once;
unticking brings it all back exactly). `mark sources` is independent of any selection: it rings
every table on the map with no incoming link and at least one outgoing one — "nothing drawn
here writes it" — in a muted grey ring, with a one-line caption under the summary strip stating
how many. A `Fed by`/`Feeds`/`Dashboards` panel section a direction mode hides keeps its heading
and its count (a fact about the graph, unchanged by the view) and says once why its rows are not
listed.

**BI items draw at three grains — workbook, tab or chart — and a table's BI edge rolls up
onto whichever grain is drawn.** The facet bar's `BI` chip group — a fourth group directly
after `kind`, selected the same way table kinds are — narrows which Tableau grain(s) are on
the map (`workbooks`/`tabs`/`charts`, plus `other` for a `.sqlcg.toml`-declared consumer);
the default draws `workbooks` and `other` only. While the group is on screen, the `kind`
group's own `consumer` chip is not shown — the `BI` group is the consumer control. The
group exists only on a host that can fetch Tableau (`tableauserverclient` importable), or an
explicit `bi_populations = true` under `[sqlcg.explorer.ui]` in the indexed repo's
`.sqlcg.toml` — the file `sqlcg index` read, not one beside where `serve` runs; elsewhere
the `kind` group's `consumer` chip governs BI items exactly as it does everywhere else, and
if the graph still holds tabs or charts (which the group would have drawn) the strip's
second line names them as *not drawn* and says why in its tooltip, rather than silently
dropping them. A table's
link to a chart that is not itself drawn is redrawn onto the nearest drawn tab it sits on,
else its workbook — the summary strip's second line says so and names what was hidden. The
legend's BI entry and the strip's BI segment both read the group's current on-set, so the
word on screen (`Workbooks`, `Workbooks, tabs & charts`, …) always names exactly what is
drawn; the legend and the strip use these three words instead of the literal `dashboard`,
which means the *workbook* to a user and the *tab* to Tableau. (The panel's `Dashboards`
section still lists BI items flat, without the workbook/tab/chart nesting the grain switch
implies — the nested panel is tracked as follow-up work, not yet shipped.)

Above the summary strip, one line states the graph's own freshness — when it was indexed,
how many commits behind that is, whether the working tree was dirty, how many statements
could not be parsed, and when the BI estate was loaded; a clause appears only when it is
true. A graph built by an older `sqlcg` replaces that line with a re-index warning, and no
config key can hide that one.

**Which UI elements are shown is configurable per element.** A narrow window has room
for the map or for the tooling around it, not both, so every finished element of the
facet bar and the side panel can be switched off — and off means *absent from the page*,
never a greyed-out control, with the space going to the map:

```toml
[sqlcg.explorer.ui]
preset = "compact"     # optional: a narrow-window set, tuned for ~960x640
legend = true          # optional per-element override; wins over the preset
panel_impact = false
```

At 960×640 the `compact` preset hides eleven of them — the legend, the `Map options` button, the
summary strip, the `kind`, `BI` and `tag` chips, the `job` dropdown, `highlight depth`, the
`certainty floor`, the graph-freshness line and the `Impact` panel section — which takes the facet bar
from 141–169 px down to 30 px at 960×640 on a graph with enough schemas for the chips to use
the horizontal strip layout (a small graph whose chips fit one plain row measures 57 px there
instead) and gives the difference to the map. Search, the `schema` chips, the `Why?`
affordance and the other panel sections stay. It changes no control's *value*: the map is still
filtered exactly as it was, only the controls are gone.

Because the controls are gone, the line under the bar stops collapsing what the map is not drawing
under the word *filters* and names each cause instead —
`6 matches · 2 on the map · 2 hidden by kind (temp, cte) · 2 with no connections`, with the config
keys that bring those controls back in its tooltip. No number on screen attributes anything to a
control this view does not show. Enter selects the first match that is **drawn** (at every preset,
not just this one), and a selected node the map is not drawing says so under its name in the panel.

**Any view you are looking at has an address.** The address bar always holds the
question the screen is answering, so a view can be bookmarked, pasted into a chat message
or sent to a colleague, and the `Copy link` button in the panel header copies exactly that
URL (its label becomes `Copied` for two seconds; if the clipboard API refuses, the URL
appears in a pre-selected text box to copy by hand):

```
/?selected=<key>&q=<text>&schema=<a,b>&kind=<a,b>&pop=<workbook,tab,chart,other>&tag=<a,b>&job=<name>&mindeg=<N>&depth=<1|2|3>&dir=<up|down>&only=1&src=1&floor=<low|medium|high>&ego=1
```

A parameter sitting at its default is never written, so the common link stays short enough
to read. Multi-value facets are one comma-joined value in the on-screen chip order (a comma
inside a value is `%2C`). A malformed `depth`, `mindeg` or `ego` is ignored in silence and
its default applies. `pop` is the `BI` chip group's on-set (omitted when it equals the
default `workbook,other`, intersected with the grains actually present in the graph); with
every BI grain off the link still writes `pop=` — the one case an empty value is written
rather than the parameter dropped, since "every grain off" is a real state a copied link
must survive. `floor` is the `certainty floor` chip's own key (`any` is the default
and is never written); a level this build does not have is named in the ignored-filters
notice rather than swallowed, and the floor's report line — what it faded, out of how many —
renders on the linked page too, not only when you click a chip yourself. `dir` is the `show`
chip's mode (`up` = `fed by`, `down` = `feeds`; omitted at `both`), `only` is `hide the other
side` (omitted when unticked, or when `dir` is `both`), and `src` is `mark sources` (omitted
when off) — `pop`/`dir`/`only`/`src` are all ignored in silence, same as a malformed value,
under a preset (or a host with no Tableau extra) that removed their control (the `BI` group,
`Map options`, direction).
Selecting a table pushes a history entry; changing a filter or typing
in search replaces one — so `Back` returns you to where you were *looking*, not through
every keystroke. A link naming a key this graph does not have says so in a sentence and
offers to search for it, and a link naming filters this graph does not have names the ones
it ignored; neither ever shows a blank canvas.

**The map is fully usable without a mouse.** `Tab` reaches the canvas as a single stop;
there arrows move a cursor through the current highlight set in the same order the panel
lists it, `Enter` or `Space` selects the node under the cursor, `Home` returns the cursor to
the selected table, and `Escape` clears the selection. `/` focuses the search box from
anywhere outside a text field, and `Escape` there clears the query — a second `Escape`
moves focus to the map. Every selection change, every announced count and every empty
state is spoken through one live region, so the screen reader is told the same thing the
screen shows.

`sqlcg explorer flags` lists every element, its current value, and which layer decided
it (`default`, `host` — the `BI` group only, when `tableauserverclient` is not importable
on the serving host — `preset`, `config`, or `dependency` — hiding `Columns` also hides
`Column lineage`, since a column can only be chosen from that list). Unknown element
keys are ignored; an unknown `preset` name is reported once on stderr at startup and
every element falls back to its default. Changes are picked up on the next
`explorer serve` start. `config` always means the `.sqlcg.toml` at the **indexed corpus
root** (the same repo path `sqlcg index` and `explorer serve` were pointed at) — a
`.sqlcg.toml` dropped beside wherever `explorer flags`/`explorer serve` happens to be run
from is not read; run `sqlcg explorer flags <corpus root>` to check the same file the
served page reads.

Opening the served UI with no query string talks to that same server's `/api/*`
routes (the real adapter, the shipped default).

`?adapter=mock` drives the UI against the committed `contract/fixtures/` example
payloads instead, with no server and no indexed graph — a demo/fixture path for
trying the UI's interaction model before indexing anything. It is served **from a
repo checkout**: open `src/sqlcg/explorer/assets/index.html?adapter=mock` through
any static file server rooted at the checkout. It does *not* work under
`sqlcg explorer serve`: that server only serves its own assets directory, so the
fixtures are outside what it will read and the fetch returns 404.

## Supported dialects

sqlcg is built on [sqlglot](https://github.com/tobymao/sqlglot), so other dialects
*can* be parsed in theory. Only Snowflake has been tested against a real corpus,
though — the table reflects what's actually been exercised, not what sqlglot can do.

| Dialect | Status |
|---------|--------|
| `snowflake` | ✅ Tested against a production DWH (~1,400 files) |
| `bigquery` | ⚠️ Unproven — parses via sqlglot, lineage not validated |
| `postgres` | ⚠️ Unproven — parses via sqlglot, lineage not validated |
| `ansi` | ⚠️ Unproven — parses via sqlglot, lineage not validated |
| `tsql` | ⚠️ Unproven — parses via sqlglot, lineage not validated |
| `dbt` | ⚠️ Unproven — via optional extra, lineage not validated |

Want another dialect properly supported? We'd be glad to collaborate — open an issue
with a minimal, anonymised corpus we can develop and test against. Getting Snowflake
right was real work, so a representative corpus is what makes the difference.

## Development

```bash
git clone https://github.com/Warhorze/sql-code-graph
cd sql-code-graph
uv sync --all-extras
uv run pytest tests/unit
```

## Issues

Bug reports and feature requests: [github.com/Warhorze/sql-code-graph/issues](https://github.com/Warhorze/sql-code-graph/issues)

Questions and discussion: [github.com/Warhorze/sql-code-graph/discussions](https://github.com/Warhorze/sql-code-graph/discussions)

## License

MIT
