Metadata-Version: 2.5
Name: attestation
Version: 0.2.0
Summary: Auditable research provenance: experiment runs, verifiable claims, a reading graph, and symbolic derivations — fully local
Project-URL: Homepage, https://github.com/mgoldey/attestation
Project-URL: Repository, https://github.com/mgoldey/attestation
Project-URL: Issues, https://github.com/mgoldey/attestation/issues
Project-URL: Changelog, https://github.com/mgoldey/attestation/blob/main/CHANGELOG.md
Author: Matthew Goldey
License-Expression: MIT
License-File: LICENSE
Keywords: claim-checking,knowledge-graph,local-first,mcp,reproducibility,research-provenance
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering
Requires-Python: >=3.12
Requires-Dist: defusedxml>=0.7.1
Requires-Dist: fastapi>=0.115
Requires-Dist: feedparser>=6.0
Requires-Dist: httpx>=0.27
Requires-Dist: jinja2>=3.1
Requires-Dist: langgraph>=0.2
Requires-Dist: mcp==1.28.1
Requires-Dist: numpy>=1.26
Requires-Dist: pydantic>=2.7
Requires-Dist: pypdf>=6.18.0
Requires-Dist: python-dotenv>=1.0
Requires-Dist: python-multipart>=0.0.6
Requires-Dist: scikit-learn>=1.5
Requires-Dist: sqlite-vec>=0.1.6
Requires-Dist: sympy>=1.13
Requires-Dist: tqdm>=4.66
Requires-Dist: uvicorn>=0.30
Description-Content-Type: text/markdown

# attestation

attestation makes research provenance auditable and fully local, for any
scientist who reads papers, keeps results files, and writes claims. It keeps
a personalised science feed and a knowledge graph built from what you read;
traces a number in your draft to the results file it came from and a
citation key to your bibliography; and does symbolic derivations — all
exposed to agents as MCP tools, with nothing leaving the machine. For
computational work it adds an experiment ledger: it reads the runs you
already have on disk (results files and W&B/MLflow/Sacred/DVC/Hydra
directories) and ranks the arms of a sweep with caveats. The feed,
provenance and symbolic tools assume nothing about your field; only the
ledger assumes you run experiments.

## Try it in 60 seconds

No model server, no ingest, no config — the run ledger and claim checker are
pure local computation over files that already exist. This uses the sweep in
`examples/workspace`:

```bash
git clone https://github.com/mgoldey/attestation ~/attestation && cd ~/attestation && uv sync
export ATTEST_DB=/tmp/attest-demo.db

uv run attest runs scan --root examples/workspace
uv run attest runs compare kdsweep
uv run attest claims examples/workspace/speech-distill/FINDINGS.md
```

Measured at **0.94s total** with no LLM backend reachable at all. What it says:

```
kdsweep — ranked by val_loss (lower_is_better), all arms on librispeech-100h
winner: kdsweep_t4
  caveat: the top two arms differ by 0.03 (1.4%) -- too close to call from
          these numbers alone
  caveat: each arm is a single run; no seed replication, so this ranking
          cannot separate configuration from run-to-run variance

7 claim(s): 1 contradicted, 5 supported, 1 unsupported
1 malformed
```

`attest claims` exits 1 on a contradiction (so it can gate a commit), which is
why a `&&` chain stops there.

Every tracker ranks arms. The second caveat — this ranking cannot separate
configuration from noise — is the part that earns its keep, and the
`contradicted` verdict is a number in a document that no longer matches the
artifact it came from.

## What it does

Three things for every scientist, one for computational experiments, and
one way to use all of them:

**For every scientist**

- **The feed and knowledge graph** rank a personalised reading list from
  cosine similarity plus click-trained terms, and derive a concept graph
  from the same tagging pass — no separate content pipeline. Any RSS/Atom
  source; arXiv is a default, not a requirement. See
  [docs/guides/feed.md](docs/guides/feed.md).
- **Data and citation provenance** check a number in your prose against
  the results file that produced it — a JSON or CSV under `results/` from
  any instrument, script or spreadsheet export, not only a training run —
  with one of five verdicts (`supported`, `contradicted`, `unsupported`,
  `ambiguous`, `stale`), and lint a citation key against your BibTeX or
  Zotero library. See
  [docs/guides/claims-and-citations.md](docs/guides/claims-and-citations.md).
- **Symbolic derivations** run in a sandboxed subprocess with a timeout and
  memory cap, never `eval`-ing input. See `sym.*` in
  [docs/guides/agents.md](docs/guides/agents.md).

**For computational experiments**

- **The experiment ledger** reads runs from artifacts already on disk — no
  instrumentation, no `log_metric()` calls — and ranks the
  [arms](docs/concepts.md) of a sweep with caveats rather than a silent
  verdict. Five tracker layouts (W&B, MLflow, Sacred, DVC, Hydra) are read
  as conventions of their own. This is the one part that assumes you run
  experiments — though a plain `results/` file is read the same way, so
  the claim checker traces every number through it. See
  [docs/guides/ledger.md](docs/guides/ledger.md).

**From an agent**

- All of the above are exposed as 49 MCP tools, restricted per session into
  `feed`/`provenance`/`knowledge`/`symbolic` surfaces. See
  [docs/guides/agents.md](docs/guides/agents.md) and the repo's own
  `src/attestation/skills/` (seven skills: one per agent surface plus setup, plus two write-side skills, `attestation-record` and `attestation-annotate`).

## Install

Two tiers. **The run ledger and claim checker above need only Python 3.12+
and [`uv`](https://docs.astral.sh/uv/)** — no model, no download. The feed,
tagging, and knowledge graph additionally need [Ollama](https://ollama.com/download)
running locally; `attest install` then pulls **~7.8 GB** of models
(`gemma4:e2b-it-q4_K_M` 7.2 GB + `embeddinggemma` 621 MB) and holds ~5.4 GB
resident while warm. Skip this tier entirely if you only want provenance.

```bash
uvx attestation install          # from PyPI, nothing cloned; --check diagnoses only
```

No GPU? Point `LLM_BASE_URL` at an OpenAI-compatible endpoint; `--check` confirms
it answers. [docs/guides/install.md](docs/guides/install.md): that tier,
prerequisites, and the manual steps `attest install` automates.

## Chat with it from Discord

Once `attest install` has run and [hermes-agent](https://github.com/NousResearch/hermes-agent)
is installed, the feed answers from a phone. Four steps, all outside this repo:

1. In the Discord Developer Portal create an application and a Bot, enable
   the **Server Members** and **Message Content** intents on the Bot page,
   and copy the token. Then `hermes gateway setup`, pick Discord, and paste
   the token plus your own Discord user ID (not the bot's).
2. Allowlist the feed surface for chat in `~/.hermes/config.yaml`, so each
   turn carries 2 tool schemas instead of 131:

   ```yaml
   mcp_servers:
     attestation-feed: {enabled: true}   # attest install wrote the entry, disabled
   platform_toolsets:
     discord: [attestation-feed]         # an MCP server here is an allowlist
   ```

3. `systemctl --user restart hermes-gateway`, then invite the bot with
   `https://discord.com/oauth2/authorize?client_id=<APP_ID>&scope=bot&permissions=274878286912`
   and DM it *"What are your recommendations today?"*

Measured on gemma4:e2b over a GTX 1080: a turn dropped from 54-249s at
15-29k prompt tokens to 23-28s at ~5k. The other half of that saving is
keeping the model resident, since a cold load alone was 30s. Full recipe,
Telegram variant, and the keep-alive timer in
[docs/guides/agents.md](docs/guides/agents.md#8-chat-from-discord-or-telegram).

## Golden paths

A golden path is a directory under `examples/` with a README in seven fixed
sections, a `run.sh` that runs those README commands verbatim, and its own
inputs on disk — `tests/test_golden_paths.py` runs every one whose
prerequisite is `none` and pins a line of its output, so the docs are what
the suite asserts. `examples/README.md` is the full catalogue, with a runtime
for each. Fourteen paths, grouped by prerequisite:

**None — pure local computation:**

- `agents/` — the install doctor, `attest emit`'s configs, one [surface](docs/concepts.md) over stdio
- `citations/` — a BibTeX library, a draft, one citation key that resolves nowhere
- `dvc/` — a real `dvc repro` pipeline, four `foreach` arms ranked
- `flows/` — forty items scored, every MCP tool over stdio, four MLflow arms
- `hydra/` — a real Hydra `--multirun` sweep, four arms ranked
- `mlflow/` — a real MLflow directory, four arms, one contradicted claim
- `model-servers/` — `attest ingest`/`tag` against an in-process stub server
- `molecular-ai/` — a molecular-AI library generated from real papers, synced, searched, walked by citation edge, graphed
- `ranking/` — twenty hand-built rows, `rank.rank_rows`'s classifier-only AUC beside the blended-order AUC
- `sacred/` — a real Sacred `FileStorageObserver` directory, four arms ranked
- `tensorflow/` — a real Keras/CSVLogger run, four learning-rate arms ranked
- `wandb/` — a real offline W&B run directory, four arms ranked
- `workspace/` — the ledger and claim checker, three claims wrong on purpose

**A model server at `LLM_BASE_URL`:** every path above needs no model; this
one does, to score prompts against a running LLM.

- `prompt-evals/` — the tagging prompt's dev score and the transfer gate

See `examples/README.md` for what each demonstrates, and `demos/` for six
scripts that record these surfaces on video. The most thorough is `flows/`:

```bash
uv run --group examples python examples/flows/run_all.py --offline
```

[`examples/flows/README.md`](examples/flows/README.md) explains each flow;
`examples/flows/RESULTS.md` records what the live run measured.

## Documentation

`uv run --group docs mkdocs serve` runs a browsable site over everything
below. The guides, one per question:

- [docs/guides/install.md](docs/guides/install.md) — set up with or without a model server
- [docs/guides/agents.md](docs/guides/agents.md) — MCP tools, surfaces, the skill
- [docs/guides/ledger.md](docs/guides/ledger.md) — reading and ranking runs
- [docs/guides/claims-and-citations.md](docs/guides/claims-and-citations.md) — checking a draft
- [docs/guides/feed.md](docs/guides/feed.md) — how ranking works
- [docs/guides/evals.md](docs/guides/evals.md) — how prompts are measured
- [docs/guides/testing.md](docs/guides/testing.md) — the gates and CI jobs

Plus [docs/concepts.md](docs/concepts.md) for first-ten-minutes vocabulary, the
CLI reference (`docs/reference/cli.md`) and API reference under `docs/`, design
records under `docs/superpowers/specs/`, measurement lessons
(`docs/measurement-lessons.md`), [CONTRIBUTING.md](CONTRIBUTING.md) for the
gates and conventions, and [CHANGELOG.md](CHANGELOG.md) for what's landed.

## Licence

MIT — see [LICENSE](LICENSE).
