Metadata-Version: 2.4
Name: datagraph-studio
Version: 0.2.0
Summary: Web UI for datagraph: pick a driver, database and schema, and get lineage, relationships, profiling, a dimensional model, PII and usage - with an MCP server for your AI assistant. Runs on a laptop or as a shared server with accounts.
Author: Sumit Kumar Gupta, Nitish Pradhan
License: Apache-2.0
Project-URL: Homepage, https://github.com/sumit-gupta03/datagraph-studio
Project-URL: Engine, https://github.com/sumit-gupta03/datagraph
Project-URL: Issues, https://github.com/sumit-gupta03/datagraph-studio/issues
Keywords: data-catalog,self-hosted,lineage,impact-analysis,dimensional-modelling,data-profiling,mcp,datagraph
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Web Environment
Classifier: Framework :: FastAPI
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Topic :: Database
Classifier: Topic :: Software Development :: Quality Assurance
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: NOTICE
Requires-Dist: datagraph-core[sql,yaml]>=0.10.7
Requires-Dist: fastapi>=0.110
Requires-Dist: uvicorn[standard]>=0.27
Requires-Dist: sqlalchemy>=2.0
Requires-Dist: cryptography>=42
Requires-Dist: argon2-cffi>=23
Provides-Extra: postgres
Requires-Dist: psycopg2-binary>=2.9; extra == "postgres"
Provides-Extra: mysql
Requires-Dist: pymysql>=1.1; extra == "mysql"
Provides-Extra: snowflake
Requires-Dist: snowflake-sqlalchemy>=1.5; extra == "snowflake"
Provides-Extra: bigquery
Requires-Dist: sqlalchemy-bigquery>=1.9; extra == "bigquery"
Provides-Extra: duckdb
Requires-Dist: duckdb>=0.10; extra == "duckdb"
Requires-Dist: duckdb-engine>=0.11; extra == "duckdb"
Provides-Extra: mssql
Requires-Dist: pyodbc>=5.0; extra == "mssql"
Provides-Extra: mcp
Requires-Dist: mcp>=1.2; extra == "mcp"
Provides-Extra: server
Requires-Dist: psycopg2-binary>=2.9; extra == "server"
Requires-Dist: gunicorn>=21; extra == "server"
Provides-Extra: all
Requires-Dist: psycopg2-binary>=2.9; extra == "all"
Requires-Dist: pymysql>=1.1; extra == "all"
Requires-Dist: duckdb>=0.10; extra == "all"
Requires-Dist: duckdb-engine>=0.11; extra == "all"
Requires-Dist: mcp>=1.2; extra == "all"
Requires-Dist: gunicorn>=21; sys_platform != "win32" and extra == "all"
Provides-Extra: dev
Requires-Dist: pytest>=7; extra == "dev"
Requires-Dist: httpx>=0.27; extra == "dev"
Requires-Dist: duckdb>=0.10; extra == "dev"
Requires-Dist: duckdb-engine>=0.11; extra == "dev"
Dynamic: license-file

# datagraph studio

A web UI for [datagraph](https://github.com/sumit-gupta03/datagraph). Pick a driver, a database
and the schemas you care about; the studio reads the schema, profiles the tables, works out the
relationships and the dimensional model, and shows you the result in a browser — then serves the
same thing to your AI assistant over MCP.

**On your laptop**, that is the whole setup — no login, no database of its own:

```bash
pip install "datagraph-studio[postgres]"
datagraph-studio --open
```

**For a team**, the same package runs as a shared server with accounts, roles, per-workspace
sharing, encrypted credentials and an audit log:

```bash
docker compose up -d
```

---

## What you get

**Connect** → pick from SQLite, DuckDB, PostgreSQL, MySQL/MariaDB, Snowflake, BigQuery,
Redshift, SQL Server, or paste any SQLAlchemy URL. Test the connection, see the schemas with
their table counts, tick the ones worth reading.

**Scan** → a background job you can watch: `information_schema` for tables, columns and
foreign keys; view definitions parsed for lineage; optional profiling; optional query-log usage.
Everything it runs against your database is a `SELECT`.

**Then six tabs:**

| Tab | What it answers |
|---|---|
| **Overview** | how big, what shape, where a change hurts most, what to look at first |
| **Explore** | search anything; one asset in full — columns, profiles, lineage, context pack |
| **Model** | facts with grain and measures, dimensions with SCD types, issues, ER diagram |
| **Relationships** | every foreign key and view-lineage link, and where each one came from |
| **Governance** | sensitive columns and what is downstream of them, usage, glossary, domains |
| **MCP** | the config to paste into Claude Code or Cursor, and the tools it exposes |

**MCP** → every workspace is an MCP server. Paste the config the UI gives you and your
assistant can answer *"what breaks if I change `dim_customer`?"* from the real schema instead
of guessing.

---

## The flow

```mermaid
flowchart LR
    C["CONNECT<br/>driver, credentials<br/>schemas to read"]
    S["SCAN<br/>information_schema<br/>view SQL, profiles<br/>SELECT only"]
    G["ONE GRAPH<br/>graph.json<br/>tables, columns, FKs<br/>lineage, profiles"]
    U["THE UI<br/>explore, model<br/>relationships<br/>governance"]
    M["MCP<br/>stdio for editors<br/>HTTP at /mcp<br/>read-only tools"]
    C ==> S ==> G
    G ==> U
    G ==> M
```

The graph is an ordinary file. The UI reads it, MCP reads it, and so does the `datagraph`
command line — the studio is a front end over the library, not a reimplementation of it.

---

## Two ways to run it

|  | Laptop | Shared server |
|---|---|---|
| Start it | `datagraph-studio` | `--auth local --database postgresql://…` |
| Sign in | no login | accounts, or your existing SSO |
| Metadata | a SQLite file | PostgreSQL |
| Credentials | in memory, never written | encrypted at rest, or per-user |
| Who sees a workspace | you | its owner, plus who they share it with |
| Audit | — | every sign-in, scan and share |

Nothing about the analysis changes. It is the same scan, the same graph and the same MCP tools;
what server mode adds is knowing who is asking.

---

## Install

```bash
pip install datagraph-studio                  # SQLite, plus any driver you already have
pip install "datagraph-studio[postgres]"      # + psycopg2
pip install "datagraph-studio[mysql]"         # + pymysql
pip install "datagraph-studio[snowflake]"     # + snowflake-sqlalchemy
pip install "datagraph-studio[bigquery]"      # + sqlalchemy-bigquery
pip install "datagraph-studio[duckdb]"        # + duckdb
pip install "datagraph-studio[mcp]"           # + the HTTP MCP transport
pip install "datagraph-studio[all]"           # the common set
```

The driver picker shows which packages are missing and the exact command to install them, so
you can start with nothing and add engines as you need them.

## Run

```bash
datagraph-studio                    # http://127.0.0.1:8770
datagraph-studio --open             # and open a browser
datagraph-studio --port 9000
datagraph-studio --home ./studio    # keep workspaces beside the project instead of in ~
```

---

## Running it for a team

The pattern is the one Grafana and Superset use: a relational **metadata database** holding
users, roles and connection definitions; credentials **encrypted at rest** with a server-side
key; and **TLS terminated by a reverse proxy** rather than by the app.

### With Docker

```bash
cp .env.example .env          # fill in the two required values
docker compose run --rm studio datagraph-studio genkey   # -> DATAGRAPH_STUDIO_SECRET_KEY
docker compose up -d
docker compose logs studio | grep -A4 "administrator account"
```

That brings up the studio, a PostgreSQL metadata database, and (with `--profile tls`) nginx.
The first run creates an `admin` account with a **generated** password, printed once, which you
are made to change at first sign-in.

### Without Docker

```bash
pip install "datagraph-studio[server,postgres]"
export DATAGRAPH_STUDIO_SECRET_KEY="$(datagraph-studio genkey)"

datagraph-studio --auth local --host 0.0.0.0 \
                 --database postgresql+psycopg2://studio:pw@db/studio \
                 --secure-cookie --workers 4
```

`--workers` needs PostgreSQL: SQLite cannot take concurrent writers, and the studio refuses
rather than corrupting itself.

**A note on `Host` checking.** On loopback the studio rejects a request whose `Host` it does not
recognise — that is the DNS-rebinding guard, and it matters on a laptop. Off loopback it cannot
know its own public name (a container publishes a different port, a proxy serves a domain), so
the check is off by default. If you do know the name, turn it back on:

```bash
datagraph-studio --host 0.0.0.0 --allowed-host studio.internal
```

### Managing accounts

```bash
datagraph-studio --home /data users              # who exists
datagraph-studio --home /data adduser alice --role editor
datagraph-studio --home /data passwd alice       # also signs their sessions out
```

or from the **Administration** page, which also shows the audit log.

**Roles.** `admin` manages people and sees everything · `editor` creates connections and
workspaces, scans, and shares what they own · `viewer` reads what has been shared with them and
cannot reach a database at all.

### Using your existing SSO

If you already have Okta, Entra, Google or anything else in front of your internal tools, do
not add another user list:

```bash
datagraph-studio --auth proxy --trusted-proxy 10.0.0.0/8 --secure-cookie
```

Your proxy authenticates and passes `X-Forwarded-User`; the studio creates the account on first
sight. It trusts that header **only** from an address you listed — a trusted header is a
complete authentication bypass if anything else can set it, so keep the studio bound to
loopback or an internal network and never publish its port. `deploy/nginx.conf` has a worked
example.

### Connections: who supplies the password

A saved connection works one of two ways, and the choice is Superset's `impersonate_user`
question:

- **Stored credential** — an admin enters a read-only service account once; it is Fernet-
  encrypted with `DATAGRAPH_STUDIO_SECRET_KEY` and everyone scans through it. Simple, and right
  when everyone would see the same thing anyway.
- **Per-user credential** — nothing is stored; each person supplies their own password, held in
  memory for their session and dropped when they sign out. Right when your warehouse's own
  grants are per-person, because then the studio cannot become a way around them.

Without a key set, the studio **refuses** to store a credential rather than writing it in the
clear.

---

## Where things are kept

A **workspace** is one database: how to reach it, and everything the last scan produced. Its
metadata lives in the database and its artefacts on disk — the same split Grafana and Superset
make, because the graph is large and rewritten wholesale by every scan.

```
~/.datagraph-studio/<workspace-id>/
    workspace.json      driver, host, schemas, when it was scanned — never a password
    graph.json          the graph: the input to the UI, to MCP and to the CLI
    relationships.json  foreign keys and lineage
    model.json          facts, dimensions, issues
    MODEL.md            the same, written for humans
    er-diagram.mmd      Mermaid ER diagram
    lineage.html        the standalone interactive graph view
    wiki/               one Markdown page per asset
```

These are ordinary files. Commit them, diff them between scans, or point the CLI at one:

```bash
datagraph context dim_customer --graph ~/.datagraph-studio/shop-a1b2c3/graph.json
```

---

## Credentials

The studio connects to production databases, so it is worth being precise about this.

- **Passwords are never written to disk.** They live in memory for as long as the process runs.
  `workspace.json` holds the driver and the non-secret fields, and a DSN with the password
  replaced by `***`. Restart the studio and a workspace still opens — the graph is on disk —
  but a *rescan* asks for the password again.
- **Read-only.** Every statement is a `SELECT` against a system view or an aggregate over a
  table. There is no query box, and no endpoint that runs SQL you type.
- **Values that look personal are masked before they are stored.** Profiling a column named
  `email` records the null rate and the distinct count, not the addresses.
- **Errors are scrubbed.** Drivers vary in how loudly they echo a connection URL back in an
  exception; the password is removed before anything reaches your browser.

### Reachability

Bound to `127.0.0.1` by default. That alone is not enough for a local app holding credentials,
so three checks run on every request:

1. **Host header** — the request must be addressed to a name this process answers to. Stops DNS
   rebinding, where an attacker's domain resolves to `127.0.0.1`.
2. **Origin** — a cross-site request that announces an origin is refused.
3. **`X-Studio: 1`** on anything that mutates — a header a cross-site form post cannot set
   without a preflight this app never approves.

Pass `--host` and a token is generated and required in `X-Studio-Token` on every request. There
is no TLS: put it behind a reverse proxy if the network is not one you trust.

---

## MCP

Open the **MCP** tab of a workspace and you get a config block to paste:

```json
{
  "mcpServers": {
    "datagraph-shop-a1b2c3": {
      "command": "datagraph",
      "args": ["mcp", "--graph", "/home/you/.datagraph-studio/shop-a1b2c3/graph.json"]
    }
  }
}
```

That launches `datagraph`'s own stdio server against the workspace's graph — it keeps working
after the studio is closed, and it holds no connection and no credential.

The studio also serves **streamable HTTP** at `/mcp` for clients that connect to a URL. Tools
take an optional `workspace` argument and default to the most recently scanned one, so one
endpoint covers every database you have opened.

| Tool | Answers |
|---|---|
| `workspaces` | the databases this studio has scanned |
| `search` | find assets by name, column, owner, tag, term or domain |
| `context` | one asset in full: columns, profile, lineage, risk, SQL |
| `lineage` | upstream and downstream trees |
| `impact` | blast radius, risk, owners and a test plan for a change |
| `relationships` | foreign keys and view lineage — the join map |
| `model` | facts, dimensions, issues and an ER diagram |
| `hotspots` | where a change hurts most |
| `sensitive_data` | where personal data lives and what is exposed to it |
| `usage` | what is actually queried, and what nobody queries |
| `paths` | how a change propagates from A to B |

All read-only, answered from `graph.json`. The server tells the model that every name, comment
and fragment of SQL it returns was copied out of your systems and must be treated as data, never
as instructions.

---

## The API

The UI has no privileged access — it is a client of the same HTTP API, documented at
`/api/docs` while the studio is running.

```
GET  /api/auth/me                          who is calling, and what they may do
POST /api/auth/login                       sign in (server mode)
GET  /api/auth/users                       manage people (admin)
GET  /api/auth/audit                       who did what (admin)
GET  /api/connections                      saved databases this user may use
POST /api/connections                      save one; the password is encrypted
GET  /api/drivers                          what can be connected to, and what is installed
POST /api/connect/test                     prove a connection works
POST /api/connect/schemas                  catalogs and schemas with table counts
POST /api/workspaces                       create, and start the first scan
POST /api/workspaces/{id}/scan             rescan
GET  /api/jobs/{id}                        progress and log of a running scan
GET  /api/workspaces/{id}/overview         the dashboard
GET  /api/workspaces/{id}/search?q=        ranked search
GET  /api/workspaces/{id}/asset?id=        one asset in full
GET  /api/workspaces/{id}/lineage?id=      upstream and downstream
GET  /api/workspaces/{id}/impact?id=       blast radius
GET  /api/workspaces/{id}/model            the dimensional model
GET  /api/workspaces/{id}/pii              sensitive data
GET  /api/workspaces/{id}/usage            query-log usage
GET  /api/workspaces/{id}/files/{name}     download an artefact
POST /api/workspaces/{id}/shares           share it with someone, or everyone
```

Embedding it in your own app:

```python
from datagraph_studio.app import create_app
from datagraph_studio.config import Settings

app = create_app(Settings(host="0.0.0.0", port=8770, token="..."))
```

---

## How it is built

No build step, no `node_modules`. The front end is ES modules and one stylesheet, served as
static files; `pip install` is the whole toolchain.

```
src/datagraph_studio/
    drivers.py       the driver registry: fields, packages, DSN assembly
    introspect.py    catalogs and schemas, before anything is scanned
    models.py        the metadata schema: users, connections, workspaces, grants, audit
    db.py            opening that database, and migrating it
    crypto.py        Fernet for stored credentials, argon2 for login passwords
    auth.py          sessions, the three auth modes, and the permission rules
    workspace.py     the workspace store and the graph cache
    jobs.py          background jobs with progress, log and cancel
    scan.py          the pipeline: extract, profile, enrich, derive
    mcp_app.py       the MCP server and the configs the UI hands out
    security.py      Host, Origin and anti-CSRF checks
    api/             the HTTP surface
    web/             index.html, app.css, js/ — no framework, no CDN
```

Adding an engine means adding one `Driver` to `drivers.py`; the front end renders whatever the
registry declares.

```bash
git clone https://github.com/sumit-gupta03/datagraph-studio
cd datagraph-studio
pip install -e ".[dev]"
pytest
```

---

## Licence

Apache-2.0 — see [LICENSE](LICENSE) and [NOTICE](NOTICE).

The engine it is built on, [datagraph-core](https://github.com/sumit-gupta03/datagraph), is
MIT-licensed and used as an ordinary dependency.

Sumit Kumar Gupta · Nitish Pradhan
