Metadata-Version: 2.4
Name: hermes-flight-recorder
Version: 0.1.0
Summary: Privacy-first, in-process JSONL flight recorder and OTLP exporter for LLM agents (built for and used by the Hermes DataForge agent).
Project-URL: Homepage, https://github.com/growthsystemes/hermes-flight-recorder
Project-URL: Documentation, https://github.com/growthsystemes/hermes-flight-recorder#readme
Project-URL: Source, https://github.com/growthsystemes/hermes-flight-recorder
Author: DataForge / GrowthSystemes
License: MIT
License-File: LICENSE
Keywords: agent,dataforge,hermes,observability,opentelemetry,otlp,tracing
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: System :: Monitoring
Requires-Python: >=3.10
Provides-Extra: otlp
Requires-Dist: httpx<1,>=0.24; extra == 'otlp'
Description-Content-Type: text/markdown

# hermes-flight-recorder

Privacy-first, in-process flight recorder for the Hermes DataForge agent and
other Python agents that want the same local observability contract.

Hermes Flight Recorder is a local-first black box recorder for agents:
canonical JSONL, privacy-safe traces, replay, explain, query, redact-check, and
optional OTLP export. It is not a dashboard; JSONL is the local source of truth.

It answers one operational question for an agent run:

> What did the agent receive, decide to call, execute, modify, cost, and what
> did the environment allow or block?

## Features

- Canonical append-only JSONL event schema.
- Optional OTLP/HTTP export without the OpenTelemetry SDK.
- Metadata-first redaction with HMAC correlation.
- W3C trace context helpers and OpenInference OTLP projection.
- Local replay, timeline, query, explain, redact-check, and doctor CLIs.
- Disabled by default; `metadata` mode writes no raw payloads.

## Install

```bash
pip install hermes-flight-recorder
```

The core package has no runtime dependencies.

For OTLP/HTTP export, install the optional transport extra:

```bash
pip install "hermes-flight-recorder[otlp]"
```

The recorder does not depend on the OpenTelemetry SDK.

## Quickstart

No Hermes, no Docker, no Kubernetes required - the core package has zero
runtime dependencies, so this runs anywhere Python 3.10+ runs:

```bash
pip install hermes-flight-recorder
python examples/minimal-demo.py
hermes-fr timeline events.jsonl --summary
```

`examples/minimal-demo.py` wraps a generic made-up agent loop (a fake tool
call plus a fake LLM call) with no external services and no network calls -
it is intentionally agent-agnostic, since the library itself has no
dependency on Hermes or DataForge (it just happens to be built inside and
used by the Hermes DataForge agent in production).

## Library Usage

```python
from hermes_flight_recorder import FlightRecorder, FlightRecorderSettings

recorder = FlightRecorder(
    FlightRecorderSettings(enabled=True, path="events.jsonl", capture_mode="metadata")
)

recorder.record_tool_call(
    tool_name="get_current_weather",
    arguments={"city": "Lisbon"},
    result={"temperature_c": 21, "conditions": "clear"},
    status="ok",
    run_id="demo",
)
```

`FlightRecorder` is exported as a neutral alias of `HermesFlightRecorder`.
`HermesFlightRecorder.from_env()` / `FlightRecorder.from_env()` reads
`FLIGHT_RECORDER_*` variables directly for non-Hermes adopters.

## Public API

The stable public surface is the top-level `hermes_flight_recorder` export:

- `HermesFlightRecorder`
- `FlightRecorder`
- `FlightRecorderSettings`
- `utc_now_iso`
- `event_to_otlp_span`
- `redact_value`
- `trace_context_payload`
- `SCHEMA_VERSION`
- `RECORDER_VERSION`
- `SEMCONV_VERSION`
- `OTEL_MAPPING_VERSION`
- `OPENINFERENCE_MAPPING_VERSION`
- `__version__`

Other functions in submodules and `_`-prefixed attributes are internal. The
canonical JSONL event schema is the stable contract. OTLP/OpenInference is a
best-effort projection and is versioned separately.

## CLIs

Installed console scripts:

```bash
hermes-fr   --version
hermes-fr   timeline events.jsonl
hermes-fr   explain events.jsonl
hermes-fr   explain events.jsonl --run <run-id>
hermes-fr   redact-check events.jsonl
hermes-fr   doctor --check-privacy events.jsonl
fr-replay   --input fixture.json --output events.jsonl --capture-mode metadata
fr-timeline events.jsonl --summary --redaction-report --structural-report
fr-query    events.jsonl --rebuild-index --summary --json
fr-explain  events.jsonl --run <run-id>
```

Equivalent module forms are available, for example:

```bash
python -m hermes_flight_recorder.flight_recorder_timeline events.jsonl --summary
```

Local verification flow:

```bash
hermes-fr replay --input examples/policy-deny.json --output events.jsonl --capture-mode metadata
hermes-fr timeline events.jsonl --show-policy --show-hashes
hermes-fr explain events.jsonl
hermes-fr explain events.jsonl --run fixture-policy-deny
hermes-fr redact-check events.jsonl
hermes-fr doctor --check-otlp --events events.jsonl --payload-out otlp.json
```

## Privacy Model

In `metadata` mode, sensitive tool arguments, results, prompts, responses,
URLs, paths, and side-effect targets are never written raw. They are stored as
keyed HMAC digests so the same value can correlate across events without being
exposed.

Configure a strong per-environment key with `FlightRecorderSettings`:

```python
recorder = HermesFlightRecorder(FlightRecorderSettings(
    enabled=True,
    capture_mode="metadata",
    hash_strategy="hmac",
    hash_secret_env="FLIGHT_RECORDER_HMAC_KEY",
    require_strong_secret=True,
))
```

If `require_strong_secret=True` and the key is missing, the recorder disables
itself instead of emitting weakly keyed hashes. The status field
`weak_secret_blocked` reports that condition.

## OTLP Export

OTLP export is disabled unless the recorder is enabled and an endpoint is set.
Install `hermes-flight-recorder[otlp]` before calling `flush_otlp()` without a
custom HTTP client. If `httpx` is missing, `flush_otlp()` fails open, leaves the
buffer intact for retry, and returns an install hint in the error payload.

The exporter sends OTLP/HTTP JSON directly:

```python
recorder = HermesFlightRecorder(FlightRecorderSettings(
    enabled=True,
    otlp_enabled=True,
    otlp_endpoint="http://collector:4318/v1/traces",
    otlp_service_name="my-agent",  # defaults to "hermes-flight-recorder" if unset
))
```

The projection includes:

- `gen_ai.*` model/tool attributes where available.
- `mcp.*` transport attributes for MCP calls.
- `openinference.span.kind`.
- `graph.node.id`, `graph.node.name`, `graph.node.parent_id`.
- `w3c.traceparent`.
- `hermes.*` privacy, runtime policy, and event metadata.

Previews are excluded from OTLP unless explicitly enabled with
`otlp_include_previews=True`.

## Rotation and Retention Presets

These are human presets for `rotate_bytes` / `retention_files`; configure them
through `FlightRecorderSettings` or the equivalent environment variables.

| Preset | rotate_bytes | retention_files | Intent |
|---|---:|---:|---|
| `local-dev` | 16 MiB | 2 | Small laptop-safe local logs. |
| `ci` | 8 MiB | 1 | Short-lived fixture and smoke output. |
| `staging` | 64 MiB | 3 | Bounded canary and soak evidence. |
| `forensic` | 256 MiB | 16 | Longer local chain for incident review. |

## Versioning

Package version `0.1.0` matches `RECORDER_VERSION=0.1.0` — this is the public
PyPI release number, which restarts independently of internal pre-publication
iteration (see `CHANGELOG.md`).
`SCHEMA_VERSION=0.3.0` remains stable for this consolidation.

The JSONL schema is the durable contract. Compatibility within 0.3.x is
additive: readers accept unknown fields, and the SQLite index is disposable and
can be rebuilt from JSONL.

The 0.3 schema was validated on DataForge staging on 2026-06-25:

- 199 metadata-only canary events.
- Redaction: 0 raw payload fields, 0 preview fields, 0 possible secret patterns.
- Structural report: 0 invalid events, 0 duplicate span IDs, 0 dangling parents.
- 0.3 gates: `mcp.tools.snapshot`, `eval.score`, and OTLP projection keys present.
- Live OTLP soak: 0 buffered events, 0 dropped events, 0 OTLP failure.

## Status

Pre-1.0. The package is regenerated from the staging-validated Hermes runtime
and is suitable for private index soak before a public PyPI release.

## License

MIT - see `LICENSE`.
