Metadata-Version: 2.4
Name: opentraces
Version: 0.3.2
Summary: Crowdsource agent traces to HuggingFace Hub
Project-URL: Homepage, https://opentraces.ai
Project-URL: Repository, https://github.com/JayFarei/opentraces
License-Expression: MIT
License-File: LICENSE
Requires-Python: >=3.10
Requires-Dist: click>=8.0
Requires-Dist: flask>=3.0
Requires-Dist: huggingface-hub>=0.20.0
Requires-Dist: mmh3>=4.0
Requires-Dist: opentraces-schema<1.0.0,>=0.3.0
Requires-Dist: pyclack-cli>=0.4.0
Requires-Dist: requests>=2.31.0
Requires-Dist: textual>=0.50.0
Provides-Extra: dev
Requires-Dist: pytest-cov; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff; extra == 'dev'
Provides-Extra: import
Requires-Dist: datasets>=2.16.0; extra == 'import'
Provides-Extra: release
Requires-Dist: build; extra == 'release'
Requires-Dist: twine; extra == 'release'
Provides-Extra: tui
Requires-Dist: textual>=0.50.0; extra == 'tui'
Provides-Extra: web
Requires-Dist: flask>=3.0; extra == 'web'
Description-Content-Type: text/markdown

# opentraces

Open schema + CLI for collecting, reviewing, and publishing agent traces to Hugging Face Hub.

Every coding session leaves behind the data you actually want: prompts, tool calls, reasoning, edits, outcome signals, and eventually the code that shipped. opentraces captures that locally, runs layered security passes, lets you review it, and publishes sharded JSONL datasets in an open schema.

> Sharing traces can leak secrets, credentials, internal paths, or customer data. opentraces reduces that risk, but it does not remove it. Read the [security docs](https://opentraces.ai/docs/security/tiers) before you publish anything.

## What It Does

1. Capture traces from supported agents such as Claude Code.
2. Enrich them with task, model, token, dependency, and git metadata.
3. Run regex, entropy, optional TruffleHog, and optional LLM review passes.
4. Stage traces locally for review in the terminal, browser, or CLI inbox.
5. Publish them as immutable JSONL shards to a Hugging Face dataset.
6. Correlate traces to later commits via `blame` and `graph`, with evidence-tiered `GitLink` attribution (`tool_emitted`, `overlapping`, `orphan`), powered by an optional background watcher.
7. Export staged traces to downstream formats such as `atif` and `agent-trace`.
8. Import existing datasets with `opentraces pull --parser hermes`, routed through the same security and staging flow.

## Install

Preferred end-user install:

```bash
pipx install opentraces
```

Homebrew:

```bash
brew install JayFarei/opentraces/opentraces
```

skills.sh (installs the opentraces skill so your coding agent can drive the workflow):

```bash
npx skills add jayfarei/opentraces
```

From source:

```bash
git clone https://github.com/JayFarei/opentraces
cd opentraces
python3 -m venv .venv
source .venv/bin/activate
pip install -e packages/opentraces-schema
pip install -e ".[dev]"
```

Use plain `pip install opentraces` only in CI or disposable environments.

Both review surfaces (`opentraces web`, `opentraces tui`) ship in the default install, no extras required.

## Quick Start

opentraces has a two-phase bootstrap: `setup` wires the machine once, `init` wires each repo.

```bash
# one-time machine setup (capture hooks, watcher, HF login, optional tiers)
opentraces setup

# initialize this repo (agents, review policy, remote dataset)
opentraces init

# review traces locally
opentraces web
# or: opentraces tui

# stage reviewed traces for upload
opentraces add --all

# publish the staged set
opentraces push
```

`init` writes the committable marker at `.opentraces.json`. Captured traces, runtime state, and upload bookkeeping stay machine-local under `~/.opentraces/projects/<slug>/`.

Useful follow-ups:

- `opentraces doctor` checks auth, integrations, and pipeline health.
- `opentraces blame <sha>` and `opentraces graph` show commit-to-trace attribution (run `opentraces setup git` first to install the post-commit correlator).
- `opentraces setup trufflehog` enables Tier 1.5 scanning.
- `opentraces setup llm-review` configures Tier 2 semantic review.
- `opentraces push --llm-review` gates uploads on a clean Tier 2 verdict.
- `opentraces assess` scores trace quality locally or across a remote dataset.
- `opentraces pull owner/dataset --parser hermes` imports traces from an existing Hugging Face dataset.
- `opentraces log` lists recent pushes grouped by date; `-v` expands to per-trace rows with tokens and cost.
- `opentraces resume <trace-id>` reopens the upstream agent session behind a trace.

## Tell Your Agent

Paste this into your coding agent:

~~~
Set up opentraces in this project.

1. Check whether `opentraces --version` works.
   If not, install with `pipx install opentraces`.

2. Run the one-time machine setup:
   `opentraces setup`

   This walks each integration (capture hooks, watcher, entity parser,
   HuggingFace login, optional TruffleHog, optional LLM review).

3. Confirm authentication:
   `opentraces auth whoami`
   If unauthenticated, use browser login (`opentraces auth login`)
   or token login (`opentraces auth login --token`).

4. Initialize the repo:
   `opentraces init`
   This prompts for agents, review policy, and the HuggingFace remote.

5. After init, the daily workflow is:
   - `opentraces status`
   - `opentraces web` or `opentraces tui`
   - `opentraces add --all`
   - `opentraces push`

6. Optional hardening:
   - `opentraces doctor`
   - `opentraces setup trufflehog`
   - `opentraces setup llm-review`
   - `opentraces push --llm-review`

7. Attribution queries (run `opentraces setup git` once to install the post-commit correlator):
   - `opentraces blame <sha>`
   - `opentraces graph`

8. Import from an existing dataset:
   - `opentraces pull owner/dataset --parser hermes`
~~~

## Security

The built-in pipeline is versioned independently from the CLI and schema (currently `SECURITY_VERSION = 0.3.0`). Run `opentraces doctor --security` to see the exact tiers, versions, and commands active in your install.

| Tier | Name | Status | What it does |
|------|------|--------|--------------|
| 1a | Regex patterns | always on | Built-in secret detectors for known token and key formats |
| 1b | Shannon entropy | always on | Flags high-entropy strings that look like secrets |
| 1.5 | TruffleHog | optional | Local scan for broader secret detection, findings redacted in place |
| 2 | LLM trace review | optional, on demand | Semantic review over the whole trace transcript |
| 3 | Human review | always available | Web inbox, TUI, and CLI review before upload |

See [security tiers](https://opentraces.ai/docs/security/tiers) and [scanning details](https://opentraces.ai/docs/security/scanning).

## Schema

The trace format lives in [`packages/opentraces-schema/`](packages/opentraces-schema/). Each JSONL line is one `TraceRecord`, with:

- task and agent identity
- TAO-loop steps
- tool calls and observations
- token and cost metrics
- outcome signals
- security metadata
- optional attribution and commit correlation data

The schema is a superset of ATIF and borrows ideas from Agent Trace, ADP, and OTel GenAI. Current schema version: `0.3.0`.

## Docs

| Section | Link |
|---------|------|
| Installation | https://opentraces.ai/docs/getting-started/installation |
| Authentication | https://opentraces.ai/docs/getting-started/authentication |
| Quick Start | https://opentraces.ai/docs/getting-started/quickstart |
| Commands | https://opentraces.ai/docs/cli/commands |
| Supported Agents | https://opentraces.ai/docs/cli/supported-agents |
| Troubleshooting | https://opentraces.ai/docs/cli/troubleshooting |
| Security Tiers | https://opentraces.ai/docs/security/tiers |
| Security Configuration | https://opentraces.ai/docs/security/configuration |
| Security Scanning | https://opentraces.ai/docs/security/scanning |
| Schema Overview | https://opentraces.ai/docs/schema/overview |
| Schema: TraceRecord | https://opentraces.ai/docs/schema/trace-record |
| Schema: Steps | https://opentraces.ai/docs/schema/steps |
| Outcome & Attribution | https://opentraces.ai/docs/schema/outcome-attribution |
| Schema Versioning | https://opentraces.ai/docs/schema/versioning |
| Parsing | https://opentraces.ai/docs/workflow/parsing |
| Inbox & Review | https://opentraces.ai/docs/workflow/review |
| Push | https://opentraces.ai/docs/workflow/pushing |
| Blame & Graph | https://opentraces.ai/docs/workflow/blame |
| Export | https://opentraces.ai/docs/workflow/export |
| Assess | https://opentraces.ai/docs/workflow/quality |
| Consume | https://opentraces.ai/docs/workflow/consume |
| Agent Setup | https://opentraces.ai/docs/integration/agent-setup |
| CI/CD | https://opentraces.ai/docs/integration/ci-cd |
| Post-Processor Contract | https://opentraces.ai/docs/integration/post-processor-contract |
| Contributing | https://opentraces.ai/docs/contributing/development |
| Schema Changes | https://opentraces.ai/docs/contributing/schema-changes |

## Packages

| Package | Description |
|---------|-------------|
| [`src/opentraces/`](src/opentraces/) | CLI, capture, review, publish, security, enrichment |
| [`packages/opentraces-schema/`](packages/opentraces-schema/) | Standalone Pydantic schema package |
| [`packages/opentraces-ui/`](packages/opentraces-ui/) | Shared design tokens and UI primitives |

## Project Layout

```text
packages/
  opentraces-schema/
  opentraces-ui/
src/opentraces/
  cli/
  core/
  capture/
  publish/
  enrichment/
  quality/
  security/
  clients/
web/
  viewer/
  site/
  coming-soon/
tests/
```

## Development

```bash
python3 -m venv .venv
source .venv/bin/activate
pip install -e packages/opentraces-schema
pip install -e ".[dev]"
pytest tests/ -v
```

## License

MIT
