Metadata-Version: 2.5
Name: agent-session-otel
Version: 0.2.0
Summary: Discover local Claude Code / Codex session logs and export them as OpenTelemetry GenAI-compatible traces.
Project-URL: Homepage, https://github.com/ryan-wolbeck/agent-session-otel
Project-URL: Issues, https://github.com/ryan-wolbeck/agent-session-otel/issues
Author: Ryan Wolbeck
License: MIT
License-File: LICENSE
Keywords: claude-code,codex,genai,observability,opentelemetry,otel,tracing
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Debuggers
Classifier: Topic :: System :: Logging
Classifier: Topic :: System :: Monitoring
Requires-Python: >=3.9
Provides-Extra: dev
Requires-Dist: opentelemetry-api>=1.24; extra == 'dev'
Requires-Dist: opentelemetry-exporter-otlp-proto-http>=1.24; extra == 'dev'
Requires-Dist: opentelemetry-sdk>=1.24; extra == 'dev'
Requires-Dist: pytest-cov>=4.1; extra == 'dev'
Requires-Dist: pytest>=7.4; extra == 'dev'
Requires-Dist: ruff>=0.5; extra == 'dev'
Provides-Extra: otel
Requires-Dist: opentelemetry-api>=1.24; extra == 'otel'
Requires-Dist: opentelemetry-sdk>=1.24; extra == 'otel'
Provides-Extra: otlp
Requires-Dist: opentelemetry-api>=1.24; extra == 'otlp'
Requires-Dist: opentelemetry-exporter-otlp-proto-http>=1.24; extra == 'otlp'
Requires-Dist: opentelemetry-sdk>=1.24; extra == 'otlp'
Provides-Extra: test
Requires-Dist: pytest-cov>=4.1; extra == 'test'
Requires-Dist: pytest>=7.4; extra == 'test'
Description-Content-Type: text/markdown

# agent-session-otel

Turn the session logs [Claude Code](https://claude.com/claude-code) and
[Codex CLI](https://github.com/openai/codex) already write to your local
disk into [OpenTelemetry](https://opentelemetry.io/) traces, using the
(still-evolving) [GenAI semantic conventions](https://github.com/open-telemetry/semantic-conventions-genai).

```
Claude Code session logs ──→ ClaudeCodeAdapter ──┐
                                                  ├──→ canonical events ──→ OTel spans ──→ JSON file / OTLP
Codex CLI session logs   ──→ CodexAdapter      ──┘
```

Both vendors' JSONL formats are undocumented, unstable, and evolve
between releases. This tool treats them as exactly that: two
disposable *adapters* that normalize into one small, stable, vendor-neutral
schema (`session` / `turn` / `tool` / `usage` / `error`, plus an `unknown`
catch-all). Nothing outside the two adapter modules knows Claude Code or
Codex's field names. When a vendor changes its format -- which will
happen -- unrecognized events are preserved (not dropped), and adding a
third vendor doesn't touch the canonical schema or the OTel export code.

## Why not native telemetry?

Neither tool exports OpenTelemetry today. Their session logs are replay
transcripts for their own resume/UI features, not telemetry -- different
shape, different guarantees, and no cross-vendor consistency. This tool
reconstructs *after the fact* what a trace of that session would have
looked like, so you can point your existing observability stack (or a
local JSON file, no stack required) at your actual coding-agent usage.
It does not run a web UI, does not run a database, and does not talk to
any vendor API -- it reads files already on disk and writes OTel data.

## What leaves your machine?

**By default: no prompt text, no file contents, no tool arguments/output,
no local file paths.** Every content-bearing field is replaced with a
`<redacted:len=N:sha256=...>` placeholder, and local paths (which for
Claude Code literally encode your project's absolute path into a
directory name) are collapsed to just a basename. Structural metadata --
event types, timestamps, token counts, tool *names* (not arguments),
model names -- passes through, since that's what makes the trace useful
without being sensitive.

Passing `--include-content` opts into carrying real prompt/tool/file
content through end to end -- do this only when exporting to a
destination you trust with that content. Even then, a secret-pattern
scrubber (API keys, bearer tokens, private key blocks, credentials
embedded in URLs, ...) still runs; `--include-content` opts into your
own prompts and file content, not into leaking a credential that happens
to appear in them.

See [Privacy defaults](#privacy-defaults) below for the full detail.

## Install

```bash
# `inspect` and `doctor` work with zero third-party dependencies.
pip install agent-session-otel

# `export` (either format) needs the OTel SDK:
pip install "agent-session-otel[otel]"

# `export --format otlp` additionally needs the OTLP/HTTP exporter:
pip install "agent-session-otel[otlp]"
```

## Quick start

```bash
# Sanity-check your environment and see what sessions are discoverable.
agent-session-otel doctor

# Summarize local sessions (Claude Code + Codex, redacted).
agent-session-otel inspect
```
```
claude_code  cc-a1b2c3d4
  file:        <redacted-path>/cc-a1b2c3d4.jsonl
  time range:  2026-08-01T10:00:00.000Z  ->  2026-08-01T10:14:22.000Z
  events:      session=2, turn=11, tool=6, usage=6, unknown=3
  tokens:      input=18300 output=2140
```
```bash
# Dump normalized events as JSON lines, content still redacted.
agent-session-otel inspect --json --vendor claude-code

# Export everything found as a local OTel JSON trace document.
agent-session-otel export --format json --output trace.json

# Export to a running OTel collector, including real content (opt-in).
agent-session-otel export --format otlp \
  --endpoint http://localhost:4318/v1/traces \
  --include-content
```

## Commands

### `agent-session-otel inspect`

Discovers session files and prints a human-readable summary per session
(event counts, token totals, time range). `--json` prints one normalized
event per line instead (the canonical schema -- see below).

### `agent-session-otel export`

Normalizes discovered sessions and exports them as an OpenTelemetry
trace: one root `invoke_agent <agent>` span per session, with one child
span per normalized event -- `chat` for turns, `execute_tool` for tool
calls (merged with their matching result into one span, when both are
present, so span duration reflects the tool's actual runtime), and
`asot.*`-namespaced spans for usage snapshots, diagnostic/session events,
errors, and anything preserved-but-unrecognized.

`--format json` needs no collector -- it writes a self-contained JSON
document of the spans. `--format otlp` requires the `otlp` extra and a
reachable OTLP/HTTP traces endpoint.

Re-running `export` against the same, unmodified session files produces
the same trace/span ids every time (derived from the vendor's session id
and each event's position in the file, not randomly generated), so a
backend that dedupes on (trace_id, span_id) recognizes a repeated import
instead of double-counting it. See [Idempotency](#idempotency-re-running-export).

### `agent-session-otel doctor`

Checks the Python version, whether the OTel SDK / OTLP exporter extras
are installed, and does an end-to-end discovery + parse smoke test
against your real session directories. Exits non-zero only on fatal
problems -- a missing session directory, or the OTel extras not being
installed, are warnings, not errors, since `inspect`/`doctor` don't need
them.

## Options common to `inspect` and `export`

| Flag | Description |
| --- | --- |
| `--vendor {claude-code,codex,all}` | Limit to one vendor. Default: `all`. |
| `--claude-code-root PATH` | Override the Claude Code session root (repeatable). Default: `~/.claude/projects`. |
| `--codex-root PATH` | Override the Codex session root (repeatable). Default: `~/.codex/sessions`. |
| `--session-id ID` | Limit to session(s) whose filename matches `ID` (repeatable). |
| `--include-content` | Opt-in: carry real prompt/tool/error text through instead of redacting it. |
| `--verbose` | Show a full traceback (instead of a one-line message) if something unexpected fails. |

Session roots can also be set via `AGENT_SESSION_OTEL_CLAUDE_CODE_HOME` and
`AGENT_SESSION_OTEL_CODEX_HOME`. Neither Claude Code, Codex, nor the
directories they write to are ever installed or required -- `doctor` and
`inspect` simply report zero sessions found for whichever vendor isn't
present on your machine, which is not an error.

## Canonical schema

Every adapter normalizes into `agent_session_otel.schema.NormalizedEvent`:
one of `session` / `turn` / `tool` / `usage` / `error` / `unknown`, plus
role, model, content, tool name/id/input/output, token usage, an
`extra` dict for small structural vendor-specific tags, and `raw` holding
the original vendor record for lossless preservation. `SCHEMA_VERSION`
(currently `1`, present on every exported/inspected event) bumps whenever
a field's meaning changes, so downstream consumers can detect a schema
they don't understand.

**Invariant:** vendor-specific concepts never become new top-level
schema fields. If a Claude Code or Codex event doesn't map onto one of
the five categories, it becomes `unknown` with the original record
preserved -- it is not dropped, and the schema does not grow a
Claude-Code-shaped or Codex-shaped field to accommodate it.

## OpenTelemetry mapping

Span and attribute names follow the
[GenAI semantic conventions](https://github.com/open-telemetry/semantic-conventions-genai)
wherever a convention genuinely exists for the concept being represented
(`gen_ai.provider.name`, `gen_ai.agent.name`, `gen_ai.operation.name`,
`gen_ai.conversation.id`, `gen_ai.usage.*`, `gen_ai.tool.*`,
`gen_ai.input.messages` / `gen_ai.output.messages`). **The entire GenAI
semconv is `development`-stability as of this writing** -- expect
upstream attribute renames (this project already migrated once, from the
now-retired `gen_ai.system` to `gen_ai.provider.name`).

Anything this tool needs that the spec doesn't define is namespaced
under `asot.*` rather than given an official-looking `gen_ai.*` name:
`asot.usage_snapshot` and `asot.error_event` (standalone spans for
events that don't cleanly attach to one chat/tool span), `asot.session_event`
(session-level metadata), `asot.unknown_event` (preserved-but-unmodeled
vendor events, full redacted payload in `asot.raw`), and
`asot.duplicate_events_skipped` (see Idempotency).

**Reconstructed vs. live telemetry:** these are historical traces built
from a static log file, not live instrumentation. Span kind is always
`INTERNAL`. A tool span's duration is the gap between its call and result
*as logged*, not a live-measured duration. Timestamps the vendor didn't
record are synthesized (nudged forward by 1μs from the prior event) purely
so spans stay orderable -- they are not real wall-clock times. None of
this is hidden: it's documented in `otel_export.py`'s module docstring,
which is worth reading before you alert on these traces as if they were
live data.

## Idempotency (re-running `export`)

Trace and span ids are derived deterministically from `(vendor, session
id, event position in file)` -- not randomly generated. Re-running
`export` against unchanged session files reliably reproduces the same
ids, and within a single run, an exact repeat of the same event (e.g. the
same file reachable from two overlapping `--claude-code-root` values)
is detected and exported once, with the count of skipped repeats on the
session's root span (`asot.duplicate_events_skipped`).

This is a best-effort identity scheme, not a cryptographic guarantee: a
session id collision across genuinely different content, or a file
edited/reordered in place rather than purely appended to, would not be
caught. It is keyed on the vendor's own session id plus each event's
line position, not on the local absolute file path (which isn't part of
a session's identity, and which redaction hides by default anyway).

## Privacy defaults

By default, every leaf string value that could contain user or model
content -- message text, tool arguments/output, error messages, local
file paths (including dict *keys* that are themselves paths, e.g. Claude
Code's file-backup tracking), and the equivalent fields inside preserved
raw vendor payloads -- is replaced with `<redacted:len=N:sha256=...>` (or,
for paths, collapsed to `<redacted-path>/basename`). Structural fields
(ids, timestamps, roles, event types, tool *names*, token counts, model
names) are never redacted.

`--include-content` disables that wholesale redaction and carries real
content through -- but a separate secret-pattern scrubber (AWS keys,
GitHub/Slack tokens, JWTs, bearer tokens, private key blocks, URL-embedded
credentials, common `key=`/`token=`/`password=` assignments) always
runs, opt-in or not, replacing matches with `<redacted-secret:label>`.
It is a best-effort net, not a guarantee -- it cannot catch every secret
shape, especially ones split across multiple tokens or non-standard
formats. Treat `--include-content` output as sensitive regardless.

## Filesystem safety

Discovery only ever reads `*.jsonl` files under the configured roots --
it never writes to, modifies, or deletes anything Claude Code or Codex
manages. A permission-denied subdirectory, a non-directory root, a
symlink loop, or an unresolvable `$HOME` are all handled gracefully
(skipped with a warning, not a crash); a single unreadable or malformed
*file* becomes one `unknown` event rather than aborting the whole scan.

## Supported versions and forward compatibility

There is no official spec for either vendor's session JSONL format, and
both have changed shape across releases (this project's Codex adapter was
written against a schema noticeably more complex than what was documented
in older community write-ups). Every field lookup in both adapters is
defensive (`.get()` with fallbacks, never an assumed key), and anything
that doesn't match a known shape -- a new top-level record type, a new
tool-call variant, a field that's now `null` where it used to be a string
-- becomes an `unknown` normalized event carrying the original record,
rather than raising or silently vanishing. On real Claude Code and Codex
history on the machine this was developed on, a meaningful fraction of
events fall into `unknown` today; that number going up over time as
vendors ship changes is expected, not a bug, and `agent-session-otel
doctor` reports it directly so you can see your own coverage.

## Development

```bash
git clone https://github.com/ryan-wolbeck/agent-session-otel.git
cd agent-session-otel
pip install -e ".[dev]"
pytest
ruff check src tests
```

Fixture-based tests live under `tests/fixtures/`. `tests/test_hostile_inputs.py`
specifically targets malformed/truncated/empty/oversized/non-UTF-8 input;
`tests/test_export.py` covers OTel semantics, deterministic ids, and
redaction; `tests/test_redaction.py` and the secret-scrubbing tests in
`test_export.py` are the ones to extend first if you're touching privacy
behavior. See [CONTRIBUTING.md](CONTRIBUTING.md).

## Scope

This project is a CLI that reads local JSONL files and writes OTel
traces. It does not run a web UI, does not run a database, does not
require authentication, and does not talk to any vendor API. It's meant
to sit alongside your existing observability stack, not replace it.

## Security

See [SECURITY.md](SECURITY.md) for how to report a vulnerability.

## License

MIT
