Metadata-Version: 2.4
Name: strata-notebook
Version: 0.8.0
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Rust
Requires-Dist: pyiceberg[sql-sqlite,sql-postgres]>=0.8.0
Requires-Dist: pyarrow>=18.0.0
Requires-Dist: fastapi>=0.115.0
Requires-Dist: starlette>=1.3.1
Requires-Dist: uvicorn>=0.35.0
Requires-Dist: websockets>=17.1
Requires-Dist: httpx>=0.28.0
Requires-Dist: duckdb>=1.1.0
Requires-Dist: python-multipart>=0.0.31
Requires-Dist: pydantic>=2.0.0
Requires-Dist: pydantic-settings>=2.15.0
Requires-Dist: tomli-w>=1.0.0
Requires-Dist: jsonschema>=4.0.0
Requires-Dist: infisicalsdk>=1.0.16
Requires-Dist: packaging>=24.0
Requires-Dist: filelock>=3.16.0
Requires-Dist: azure-storage-blob>=12.20.0 ; extra == 'azure'
Requires-Dist: azure-identity>=1.25.3 ; extra == 'azure'
Requires-Dist: mcp>=1.9.0 ; extra == 'mcp'
Requires-Dist: pandas>=2.0.0 ; extra == 'notebook'
Requires-Dist: numpy>=1.26.0 ; extra == 'notebook'
Requires-Dist: orjson>=3.10.0 ; extra == 'notebook'
Requires-Dist: cloudpickle>=3.0.0 ; extra == 'notebook'
Requires-Dist: matplotlib>=3.8.0 ; extra == 'notebook'
Requires-Dist: pillow>=10.0.0 ; extra == 'notebook'
Requires-Dist: opentelemetry-api>=1.20.0 ; extra == 'otel'
Requires-Dist: opentelemetry-sdk>=1.20.0 ; extra == 'otel'
Requires-Dist: opentelemetry-exporter-otlp>=1.20.0 ; extra == 'otel'
Requires-Dist: opentelemetry-instrumentation-fastapi>=0.41b0 ; extra == 'otel'
Requires-Dist: psycopg[binary]>=3.1 ; extra == 'postgres'
Requires-Dist: psycopg-pool>=3.2 ; extra == 'postgres'
Requires-Dist: sqlglot>=25.0.0 ; extra == 'sql'
Requires-Dist: adbc-driver-manager>=1.0.0 ; extra == 'sql'
Requires-Dist: adbc-driver-bigquery>=1.0.0 ; extra == 'sql-bigquery'
Requires-Dist: duckdb>=1.0.0 ; extra == 'sql-duckdb'
Requires-Dist: adbc-driver-postgresql>=1.0.0 ; extra == 'sql-postgres'
Requires-Dist: adbc-driver-snowflake>=1.0.0 ; extra == 'sql-snowflake'
Requires-Dist: adbc-driver-sqlite>=1.0.0 ; extra == 'sql-sqlite'
Requires-Dist: textual>=0.80.0 ; extra == 'tui'
Requires-Dist: grandalf>=0.8 ; extra == 'tui'
Requires-Dist: textual-image>=0.13 ; extra == 'tui'
Requires-Dist: pillow>=10.0.0 ; extra == 'tui'
Provides-Extra: azure
Provides-Extra: mcp
Provides-Extra: notebook
Provides-Extra: otel
Provides-Extra: postgres
Provides-Extra: sql
Provides-Extra: sql-bigquery
Provides-Extra: sql-duckdb
Provides-Extra: sql-postgres
Provides-Extra: sql-snowflake
Provides-Extra: sql-sqlite
Provides-Extra: tui
License-File: LICENSE
Summary: A notebook built for coding agents: every result cached, versioned, and visible to the human watching
Keywords: notebook,materialization,provenance,lineage,reproducibility,content-addressed,iceberg,arrow,analytics
Author-email: Fangchen Li <fangchen.li@outlook.com>
License-Expression: Apache-2.0
Requires-Python: >=3.12
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Changelog, https://github.com/bearing-research/strata/blob/main/CHANGELOG.md
Project-URL: Documentation, https://bearing-research.github.io/strata/
Project-URL: Homepage, https://bearing-research.github.io/strata/
Project-URL: Issues, https://github.com/bearing-research/strata/issues
Project-URL: Repository, https://github.com/bearing-research/strata

# Strata

[![PyPI](https://img.shields.io/pypi/v/strata-notebook.svg)](https://pypi.org/project/strata-notebook/)
[![Python versions](https://img.shields.io/pypi/pyversions/strata-notebook.svg)](https://pypi.org/project/strata-notebook/)
[![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/bearing-research/strata/blob/main/LICENSE)
[![CI](https://github.com/bearing-research/strata/actions/workflows/ci.yml/badge.svg)](https://github.com/bearing-research/strata/actions/workflows/ci.yml)
[![Pre-commit](https://github.com/bearing-research/strata/actions/workflows/pre-commit.yml/badge.svg)](https://github.com/bearing-research/strata/actions/workflows/pre-commit.yml)
[![Docker](https://github.com/bearing-research/strata/actions/workflows/docker.yml/badge.svg)](https://github.com/bearing-research/strata/actions/workflows/docker.yml)
[![Docs](https://github.com/bearing-research/strata/actions/workflows/docs.yml/badge.svg)](https://github.com/bearing-research/strata/actions/workflows/docs.yml)
[![codecov](https://codecov.io/gh/bearing-research/strata/branch/main/graph/badge.svg?token=GBAX34U2PO)](https://codecov.io/gh/bearing-research/strata)
[![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/bearing-research/strata/badge)](https://securityscorecards.dev/viewer/?uri=github.com/bearing-research/strata)

**Strata is a notebook your coding agent can drive, and everything it computes stays.**

Coding agents explore by writing throwaway scripts: `python -c`, files in
`/tmp`, invisible and gone by the next session. Point one at Strata and the
work lands in a notebook instead. Every result is cached by what produced it,
every cell records who wrote it, and the whole thing is open in your browser
while the agent works. Come back tomorrow and it is all still there, already
computed.

That holds because of what a cell is. Each one is keyed by its source, its
inputs and its environment, so the expensive step an agent ran ten turns ago
is a cache hit now, and the same inputs give the same result a year later on
another machine. Change one cell and only the cells below it run again.
Nothing is recomputed to prove a point.

It is a good notebook for a person, too. Ask a model a question in a prompt
cell and the answer is cached like any other result. Send the slow cell to a
GPU with one line. Read an S3 prefix as a local path. The notebook itself is
plain `.py` files plus a manifest, so a commit is a readable diff rather than
a wall of JSON.

<picture>
  <source media="(prefers-color-scheme: dark)" srcset="docs/assets/notebook-anatomy-dark.png">
  <img alt="The Strata notebook UI: a Python cell with its source and an interactive table of results, sidebar panels for mounts, workers and environment, and a bottom drawer showing the cell DAG and per-cell timings." src="docs/assets/notebook-anatomy-light.png">
</picture>

**Docs:** [bearing-research.github.io/strata](https://bearing-research.github.io/strata/)

## Point an agent at it

Install it as a one-command [Claude Code plugin](plugins/strata-scratchpad/)
(needs the `strata` CLI on `PATH`; `uv tool install strata-notebook`):

```
/plugin marketplace add bearing-research/strata
/plugin install strata-scratchpad@strata
```

![The terminal viewer over five moments: an empty notebook, a cell arriving,
that cell finishing in 1.0s with its output, a second cell arriving that reads
the first one's variable, and both cells green.](https://raw.githubusercontent.com/bearing-research/strata/main/docs/assets/tui-agent-live.gif)

*A coding agent drives the notebook in one terminal; `strata-notebook-tui`
mirrors it live in another. Captured from a real run, not a mock-up.*

Whether an agent reaches for it instead of a scratch script is measured rather
than assumed. `evals/agent_notebook/` drives the real on-ramp and grades the
result: 9 transcript-backed tasks run in CI, and a `scratchpad` group probes
whether an un-primed session uses the notebook at all. See
[Driving a notebook with a coding agent](https://bearing-research.github.io/strata/latest/notebook/agent/).

## Highlights

**For an agent**

- **a scratchpad that lasts:** a Claude Code plugin points an agent at a cached notebook cell instead of a `/tmp` script, so the snippet it ran ten turns ago is still there
- **any MCP client:** a warm notebook session is exposed at `/mcp`, and you can watch the agent work in the browser or the terminal while it does
- **built to be scripted:** `strata` inspects, runs and authors a notebook with JSON output and stable exit codes, offline or against a live session
- **one command to start:** `strata agent ./nb` stands up the server, the session, the config and the viewer together
- **you can tell who did what:** every cell records whether a person or an agent wrote it

**Why nothing runs twice**

- **results are keyed by what made them:** a cell's source, its inputs and its environment decide its identity, so identical work is a cache hit forever
- **only what changed re-runs:** edit one cell and Strata re-runs the cells below it and nothing else
- **no wiring:** Strata reads each cell to find what it uses and what it defines, so there are no decorators and no edges to maintain by hand
- **readable in git:** a notebook is plain `.py` files plus a TOML manifest, so a diff shows the code that changed
- **the graph is visible:** the dependency graph renders beside the cells, and double-clicking a node jumps to its source

**Cell types**

- **prompt cells:** LLM calls as DAG nodes, `{{ variable }}` interpolation from upstream, cached by template + inputs + model config
- **SQL cells:** named connections and bind-parameter templating, with drivers for DuckDB, SQLite, Postgres, Snowflake and BigQuery
- **R cells:** Python and R share one DAG, exchanging Arrow, so a `pandas.DataFrame` is a `data.frame` for the next cell
- **loop cells:** `# @loop max_iter=N carry=state` iterates with an explicit carry, each step its own artifact
- **variant sweeps:** run every variant of a group and hand the downstream cell a `{name: value}` dict, or fan out with `# @per_variant` so adding a variant only runs the new one
- **widget cells:** declarative controls that downstream cells consume as inputs; with Live on, dragging one recomputes what depends on it
- **cell unit tests:** a Tests panel runs real pytest against a cell's defs and upstream inputs, doubling as a health badge


**Compute and data**

- **distributed:** `# @worker gpu-fly` dispatches a cell to a remote box; bring your own compute
- **worker pool** (separate package, `pip install strata-pool`)**:** machines (Fly, Docker or RunPod) start on demand, are held for a tenant, hand out GPUs per cell, and stop when the work does
- **the same environment, elsewhere:** a remote cell runs in the notebook's own locked environment rather than whatever the worker image holds
- **remote cells over SSH:** hand an agent an SSH target and it provisions a worker there, tunnels to it, and routes heavy cells over, cached like everything else
- **lake-aware SQL:** read a named catalog and the notebook's mounts in one query, pinned to the snapshot the cell's provenance records
- **every input recorded:** `# @fetch` makes bytes from a URL an input, `# @dataset` makes a registry name one, both content-addressed
- **mounts:** `# @mount data s3://bucket/prefix ro` makes any S3, GCS or Azure prefix a local `pathlib.Path`
- **isolated envs:** every notebook gets its own uv-managed `.venv/`, locked and reproducible
- **headless:** `strata run ./my-notebook` for CI and scheduled execution, same DAG and same cache

**Teams**

- **promote to the team:** `strata artifact promote` copies a result and the chain behind it into the team's store, from the notebook, the CLI or inside a cell
- **a colleague's run counts:** a cell can be served from a teammate's earlier result: the expensive middle of a pipeline, not only what someone chose to name
- **a figure gets a URL:** publish an artifact and anyone with the link sees the plot, the code, and the environment of every step behind it, with no account and no install
- **a notebook travels:** export the whole state as one bundle and import it back, artifacts included
- **share a session:** presence, cell focus and soft locks, with every cell recording who wrote it
- **safe to share** (in service mode)**:** cells run as their own OS user once you name one, notebook scopes are enforced on every notebook route under principal auth, and an MCP tool call runs as its caller

**Surfaces**

- **terminal viewer:** `strata watch` renders a running notebook live in the terminal, read-only, over SSH or beside your editor
- **app view:** open a notebook as a read-only app, embed it as an `<iframe>`, or export a self-contained snapshot
- **data viewer:** DataFrame outputs render in a grid you can page, sort, filter and search, backed by the full artifact rather than a preview
- **also a library:** `pip install strata-client` talks to the store from any pipeline or service, with no server install
- **production:** Iceberg-aware scans, trusted-proxy auth, multi-tenancy, and S3, GCS, Azure or local blob backends

## Quick Start

Both paths below run in **personal mode**: single-user, writes enabled, no
proxy auth. For multi-tenant or hosted deployments, see
[Deployment Modes](https://bearing-research.github.io/strata/latest/deployment/modes/).

```bash
# Docker. docker-compose.yml sets personal mode for you.
docker compose up -d --build
# Then open http://localhost:8765

# Or install via uv (recommended). Puts the CLI on PATH in a uv-managed
# tool env. `pip install` is not supported; see Requirements below.
uv tool install strata-notebook
strata-notebook
# Then open http://localhost:8765
```

For the full inventory of installed commands (`strata-notebook`, `strata`,
`strata-worker`, `python -m strata`), see the
[Commands reference](https://bearing-research.github.io/strata/latest/getting-started/installation/#commands-reference).

Source builds (`git clone + uv sync`) are documented in
[Installation](https://bearing-research.github.io/strata/latest/getting-started/installation/).

### Requirements

**[uv](https://docs.astral.sh/uv/) 0.8 or newer**, and nothing else. uv fetches
a matching Python for you.

Strata runs only inside a uv-managed environment. It checks for the
`uv = <version>` marker uv writes to `pyvenv.cfg`, which `uv tool install`,
`uv add` and `uv run` all produce and a hand-rolled `python -m venv` does not.
The reason is that the notebook subsystem shells out to `uv` to manage
per-notebook `.venv/` directories; failing at startup with a clear message
beats a confusing subprocess error later. Conda and pip-venv users install uv
and relaunch from a uv-managed env, leaving existing data and environments
untouched.

Windows: `uv tool install strata-notebook` works directly.

Building Strata itself from a git clone also needs a
[Rust toolchain](https://rustup.rs/) for the native extension and
[Node 26+](https://nodejs.org/) for the frontend. PyPI wheels ship both
prebuilt, so this applies only if you are modifying Strata. Source builds work
on Windows via WSL2 or natively.

## The Cache Advantage

Every notebook platform re-executes from scratch when you change one cell.
Strata doesn't. The artifact store deduplicates by provenance hash. If
the code and inputs haven't changed, the result is served instantly.

```
First run:     load data (10s) → clean (3s) → train (20s) → evaluate (1s)  = 34s
Change model:  load data (✓)   → clean (✓)  → train (20s) → evaluate (1s)  = 21s
Re-run:        load data (✓)   → clean (✓)  → train (✓)   → evaluate (✓)   = <1s
```

This is the architecture, not a layer on top of it. Every cell execution is a
`materialize(inputs, transform, environment) → artifact` operation, and the
cache is keyed on content rather than time, so it is correct by construction.

Even a leaf cell that only `print`s is cached: its console output is keyed
by the same provenance hash, so an unchanged re-run replays the output
instantly rather than executing again. That makes the notebook a good
**scratchpad** - a coding agent's exploratory snippets stop recomputing.
Cells that must always run (a side effect, a live API call, a fresh random
draw) opt out with `# @nocache`.

## Distributed Execution

Each cell can declare which worker it runs on via a single annotation:

```python
# @worker my-gpu
embeddings = model.encode(abstracts, batch_size=256)
```

You define workers in `notebook.toml`. Each one points at an HTTP
endpoint that implements the Strata executor protocol. A worker can be
a GPU box on RunPod, a DataFusion cluster on Fly, a beefy EC2 instance,
or anything else that speaks HTTP. The notebook routes the cell to the
declared worker at execution time, and the UI shows a live
`dispatching → my-gpu` badge while it runs.

No deployment code, no infrastructure glue. Bring your own compute,
one annotation per cell.

## Source Annotations

Every piece of per-cell metadata is a comment directive in the cell's
source. The source is the single canonical place for cell config:
annotations always win over any stored defaults.

```python
# @name Extract embeddings
# @worker gpu-fly
# @timeout 600
# @env MODEL_PATH=/models/bge-large
# @mount dataset s3://corpus/2024-q4 ro
embeddings = model.encode(dataset / "abstracts.jsonl")
```

Diagnostics fire on open, reload, and after an edit settles, for example
`worker_unknown`, `mount_uri_unsupported`, `mount_shadows_notebook`,
`timeout_not_numeric`, `env_malformed`. They surface as a pill in the
cell header and log structured warnings for headless runs.

## Mounts

Mounts bind a remote URI to a local path inside the cell. Supported
schemes: `file://`, `s3://`, `gs://`, `az://`. Credentials flow through
fsspec options: set `anon = true` for public buckets, or drop it to
use the standard credential chain.

```toml
[[mounts]]
name = "taxi_zones"
uri = "s3://nyc-tlc/misc"
mode = "ro"
options = { anon = true }
```

Inside the cell, `taxi_zones` is a `pathlib.Path`. Strata materializes
it on first read and caches the bytes locally for the session.

## Examples

| Example                                             | What it shows                                                                       |
| --------------------------------------------------- | ----------------------------------------------------------------------------------- |
| [pandas_basics](https://bearing-research.github.io/strata/latest/examples/pandas_basics/)             | Linear DataFrame chain, caching, staleness propagation, per-cell unit tests          |
| [iris_classification](https://bearing-research.github.io/strata/latest/examples/iris_classification/) | End-to-end ML, DAG branching, mixed output types                                    |
| [titanic_ml](https://bearing-research.github.io/strata/latest/examples/titanic_ml/)                   | Feature engineering + model comparison                                              |
| [s3_mount](https://bearing-research.github.io/strata/latest/examples/s3_mount/)                       | Reading a public S3 bucket via a mount                                              |
| [arxiv_classifier](https://bearing-research.github.io/strata/latest/examples/arxiv_classifier/)       | Distributed execution via `@worker` + Modal GPU + Fly cluster                       |
| [markdown_showcase](https://bearing-research.github.io/strata/latest/examples/markdown_showcase/)     | Markdown cells, dynamic `Markdown(...)` outputs, security cases                     |
| [library_cells](https://bearing-research.github.io/strata/latest/examples/library_cells/)             | Cross-cell library code: pure module cells, mixed runtime+library cells, the limits |
| [news_alpha_trader](https://bearing-research.github.io/strata/latest/examples/news_alpha_trader/)     | Multi-stage trading pipeline with prompt cells and structured LLM outputs           |
| [review_triage](https://bearing-research.github.io/strata/latest/examples/review_triage/)             | Prompt cell with `# @output_schema` - structured, schema-validated LLM output       |
| [r_mtcars_analysis](https://bearing-research.github.io/strata/latest/examples/r_mtcars_analysis/)     | Pure-R notebook - `mtcars` regression with inline ggplot2 / base-graphics plots     |
| [r_lm_vs_sklearn](https://bearing-research.github.io/strata/latest/examples/r_lm_vs_sklearn/)         | Cross-language R + Python - R `lm()` vs scikit-learn over shared Arrow data          |
| [sql_orders_report](https://bearing-research.github.io/strata/latest/examples/sql_orders_report/)     | SQL cells over a local SQLite warehouse, mixing SQL / Python / markdown              |
| [widget_playground](https://bearing-research.github.io/strata/latest/examples/widget_playground/)     | Widget cell (control panel) driving a downstream cell, with `# @live` reactivity     |
| [model_variants](https://bearing-research.github.io/strata/latest/examples/model_variants/)           | Variant groups - alternative training cells sharing one DAG slot (switch mode)       |
| [model_variants_sweep](https://bearing-research.github.io/strata/latest/examples/model_variants_sweep/) | Variant sweep mode - run every variant and compare them in one downstream cell     |
| [loop_hill_climb](https://bearing-research.github.io/strata/latest/examples/loop_hill_climb/)         | `# @loop` / `# @loop_until` iteration with carry and per-iteration artifacts         |
| [data_viewer](https://bearing-research.github.io/strata/latest/examples/data_viewer/)                 | Interactive data viewer - page / sort / filter / search the full cached artifact     |
| [agent_demo](https://bearing-research.github.io/strata/latest/examples/agent_demo/)                   | A coding agent builds the notebook live; only the eval re-runs when the model caches |

## What is still moving

Materialization, the artifact store, the DAG, caching and headless run are
settled and covered by tests. Three surfaces are not:

- **Prompt-cell API.** Streaming, conversation memory and structured-output
  validation will change.
- **SQL cloud drivers.** DuckDB, SQLite and PostgreSQL run in CI. BigQuery and
  Snowflake ship without integration coverage. MotherDuck and MySQL are not
  implemented.
- **On-disk formats.** `notebook.toml`, `.strata/runtime.json` and the artifact
  cache layout may change between minor versions. Depend on the Python API,
  not the file shapes.

---

## Library usage

Strata's HTTP API exposes the materialization layer directly,
driveable from Python via `StrataClient`. Useful for direct table
scans, custom transforms, and headless workflows; the notebook
executor is a separate pipeline that writes to the same artifact
store. The client talks to a running Strata server, so this workflow
has two steps: start the server, then call it from your code.

```bash
# 1. Install + start the server (in a uv-managed env).
uv tool install strata-notebook
strata-notebook

# 2. In your own project, install the slim client - a separate package
#    (httpx + pyarrow only, no server deps, plain pip is fine) - and
#    point it at the running server:
pip install strata-client
```

```python
from strata_client import StrataClient

client = StrataClient(base_url="http://localhost:8765")
artifact = client.materialize(
    inputs=["file:///warehouse#db.events"],
    transform={"executor": "scan@v1", "params": {"columns": ["id", "value"]}},
)
table = client.fetch(artifact.uri)  # Arrow table, cached by provenance
```

The server provides: provenance-based deduplication, immutable
versioned artifacts, lineage tracking, Iceberg table scanning with
row-group caching, pluggable blob storage (local/S3/GCS/Azure),
multi-tenancy, trusted-proxy auth, and an executor protocol for
external compute.

**[Library docs →](https://bearing-research.github.io/strata/latest/getting-started/core/)**

---

## Architecture

```
┌─────────────────────────────────────────────┐
│ Notebook UI (Vue.js + WebSocket)            │
│ cells, DAG view, workers                    │
└─────────────────────────────────────────────┘
                    │
                    ▼
┌─────────────────────────────────────────────┐
│ Notebook Backend (FastAPI)                  │
│ session, cascade, executor, prompt cells    │
└─────────────────────────────────────────────┘
                    │
                    ▼
┌─────────────────────────────────────────────┐
│ Strata Core                                 │
│ materialize, artifacts, lineage, dedupe     │
└─────────────────────────────────────────────┘
```

The notebook is an orchestration layer over Core. It decides what to
run next (cascade planning, staleness tracking). The cell harness is an
executor. Core decides whether results already exist and persists them.

## Development

```bash
uv sync --all-extras                   # Install deps + build Rust extension (matches CI)
uv run pytest                          # Run all tests
uv run pre-commit run --all-files      # Lint + format
cd frontend && npm run dev             # Frontend dev server (hot reload)
```

## License

Apache 2.0

