Metadata-Version: 2.4
Name: labcoat
Version: 0.1.0
Summary: Automated research lab notebook: hypotheses, code diffs, results, and conclusions for every experiment you run.
Project-URL: Homepage, https://github.com/como-research/labcoat
Author-email: Chase McDonald <chasecmcdonald@gmail.com>
License: MIT
License-File: LICENSE
Keywords: experiments,lab-notebook,labcoat,ml,reproducibility,research
Classifier: Development Status :: 3 - Alpha
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.10
Requires-Dist: litellm>=1.40
Requires-Dist: numpy>=1.24
Requires-Dist: pydantic>=2.0
Requires-Dist: rich>=13.0
Requires-Dist: scipy>=1.10
Requires-Dist: typer>=0.12
Provides-Extra: mcp
Requires-Dist: mcp>=1.0; extra == 'mcp'
Provides-Extra: wandb
Requires-Dist: wandb>=0.16; extra == 'wandb'
Description-Content-Type: text/markdown

# labcoat

An automated research lab notebook. Experiment trackers record *what the
metrics were*; labcoat records *why you ran it, what you changed, and what you
concluded* — automatically.

```bash
labcoat run -- ./launch_seeds.sh --algo sac --env Walker2d-v4
```

Every launch captures the git state (HEAD + dirty diff as a reproducible
patch), the exact command, and a hypothesis — prompted at launch, drafted by
an LLM from your diff. Results flow in from your existing tracker (wandb
first; labcoat never reimplements logging). Trials group automatically,
statistics come with confidence intervals and significance tests, and an LLM
pass drafts conclusions and surfaces anomalies in metrics you weren't
watching.

See [docs/walkthrough.md](docs/walkthrough.md) for a full worked example
(PPO vs SAC, sweeps, multi-seed stats, the review flow).

## Principles

- **labcoat never owns execution.** It wraps whatever you already run — a
  multi-seed launcher script, a slurm submit, a sweep agent. Tracker runs
  link to the experiment through propagated environment variables; completion
  is tracked through tracker run states, so detached launchers just work.
- **Never reinvent logging.** Metrics stay in wandb/tensorboard/etc., behind
  an adapter protocol.
- **The hypothesis is the soul.** Capture is low-friction (prompt with an
  LLM draft, `-m` flag, backfill later) but always present in the record.
- **Local-first.** SQLite in `.labcoat/` (gitignored); the generated
  `labbook/` markdown is committed so your research history travels with the
  repo.

## Installation

labcoat is on [PyPI](https://pypi.org/project/labcoat/). The recommended
install is as a **uv tool**, which puts the `labcoat` command on your PATH so
you can call it directly from any repo — no `uv run` prefix, no activating
environments:

```bash
uv tool install "labcoat[wandb]"

labcoat --version           # works from anywhere
labcoat run -- python train.py
```

Upgrade later with `uv tool upgrade labcoat`.

Alternatively, add it as a dependency of your research project
(`uv add "labcoat[wandb]"`) — then it runs as `uv run labcoat` or plain
`labcoat` inside the activated environment. The tool install is the better
default: the notebook wraps your project, it doesn't need to live inside it.

## Quick start

```bash
cd your-research-repo
labcoat init                       # writes labcoat.toml, creates .labcoat/
labcoat run -- ./launch_seeds.sh   # hypothesis prompt (LLM-drafted from your diff)
labcoat log                        # timeline; lazy-syncs tracker state first
labcoat review                     # accept/edit drafted conclusions, metric proposals
labcoat compare exp-0001 exp-0002  # bootstrap CIs, Welch/Mann-Whitney, Cohen's d, Holm
labcoat show exp-0002 --diff       # full record incl. the captured patch
labcoat search "entropy collapse"  # months-later recall
labcoat book                       # regenerate committed labbook/ markdown
labcoat mcp                        # serve the record to coding agents (MCP)
```

## Commands

`init` · `run` (`-m` hypothesis, `-y` accept drafts, `--group`, `--pin` shadow
commit) · `log` · `show` · `compare` · `review` · `sync` · `search` · `book` ·
`mcp` · `annotate` · `conclude` · `exclude` · `group`

## LLM configuration

The LLM drafts hypotheses from your diffs, summarizes changes, groups runs
semantically, narrates discovery findings, and drafts conclusions. It is
configured in `labcoat.toml` (any [litellm](https://docs.litellm.ai/docs/providers)
model string) with credentials from the environment — labcoat never stores keys:

```toml
[llm]
model = "anthropic/claude-sonnet-4-6"
context_budget = 12000     # max tokens of experiment history per call
```

```bash
export ANTHROPIC_API_KEY=sk-ant-...   # or OPENAI_API_KEY, GEMINI_API_KEY, LITELLM_API_KEY
```

**Local models via ollama** — no key needed; just point at a local model:

```toml
[llm]
model = "ollama/llama3.1"
# api_base = "http://gpu-box:11434"   # only if ollama isn't on localhost:11434
```

`api_base` also works for any OpenAI-compatible server (vLLM, LM Studio):

```toml
[llm]
model = "hosted_vllm/meta-llama/Llama-3.1-70B-Instruct"
api_base = "http://localhost:8000/v1"
```

Without any of these, every LLM feature degrades gracefully: launches never
block, raw diffs are stored instead of summaries, grouping falls back to
byte-identity, and conclusions are written by hand in `labcoat review`. Set
`enabled = false` under `[llm]` to turn the LLM off explicitly, and
`labcoat exclude exp-NNNN` to keep any experiment out of LLM context.

## Development

The project is managed with [uv](https://docs.astral.sh/uv/). From a checkout:

```bash
uv sync                 # creates .venv from uv.lock (includes the dev group)
uv run pytest           # run the test suite
uv run labcoat --help   # run the CLI from source
```

Extras: `uv sync --extra wandb --extra mcp` to develop against the adapters.

Releases go to PyPI with:

```bash
uv build                # sdist + wheel into dist/
uv publish              # needs a PyPI token (UV_PUBLISH_TOKEN)
```

## Status

v0.1: the full loop works — launch capture (git patch snapshots, optional
shadow commits), LLM diff summaries + hypothesis drafts (litellm; degrades
gracefully without a key), wandb ingestion with adapter-state-driven lazy
sync, semantic auto-grouping, sweep arms auto-derived from run configs, the
statistics toolkit, the discovery pass over all logged series, drafted
conclusions with a review flow, committed `labbook/` rendering, search, and
an MCP server. Planned next: tensorboard/mlflow adapters and remote sync of
the record.
