Metadata-Version: 2.4
Name: traceguard
Version: 0.9.0
Summary: Point-in-time correct LLM instrumentation: tracing, version pinning, and look-ahead-bias protection for research pipelines
Project-URL: Homepage, https://github.com/lizhuojunx86/traceguard
Project-URL: Repository, https://github.com/lizhuojunx86/traceguard
Project-URL: Documentation, https://github.com/lizhuojunx86/traceguard/blob/main/docs/SPEC.md
Author: Li Zhuojun
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: backtesting,data-contamination,llm,llm-evaluation,look-ahead-bias,observability,point-in-time,prompt-versioning,reproducibility,temporal-integrity,tracing
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Financial and Insurance Industry
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Quality Assurance
Requires-Python: >=3.11
Requires-Dist: pydantic>=2.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: sqlalchemy>=2.0
Provides-Extra: anthropic
Requires-Dist: anthropic>=0.40.0; extra == 'anthropic'
Provides-Extra: contamination
Provides-Extra: contamination-hf
Requires-Dist: torch>=2.0; extra == 'contamination-hf'
Requires-Dist: transformers>=4.40; extra == 'contamination-hf'
Provides-Extra: openai
Requires-Dist: openai>=1.0; extra == 'openai'
Provides-Extra: otel
Requires-Dist: opentelemetry-api>=1.20; extra == 'otel'
Requires-Dist: opentelemetry-exporter-otlp-proto-http>=1.20; extra == 'otel'
Requires-Dist: opentelemetry-sdk>=1.20; extra == 'otel'
Description-Content-Type: text/markdown

# traceguard

**Point-in-time correct LLM instrumentation — the time-integrity layer for
LLM pipelines.**

When you run LLMs over historical data — backtesting a signal, replaying a
pipeline, re-scoring an archive — TraceGuard makes it structurally hard to
accidentally use a model or prompt that did not yet exist at the point in
time you are simulating.

It is not a dashboard or a gateway; it is the lower layer that guarantees the
timeline underneath one. It interoperates with observability stacks
(Langfuse / Phoenix via the optional `traceguard[otel]` exporter) rather than
competing with them — see the
[OpenTelemetry → Langfuse/Phoenix guide](https://github.com/lizhuojunx86/traceguard/blob/main/docs/integrations/otel-langfuse-phoenix.md)
and [docs/POSITIONING.md](https://github.com/lizhuojunx86/traceguard/blob/main/docs/POSITIONING.md).

- `traceguard.registry.models` — model registry with `released_at` /
  `available_to_us_at`; `select_model(..., strict=...)` with mandatory
  explicit mode (no default), so anachronistic choices fail loudly.
- `traceguard.registry.prompts` — git-tracked YAML prompt templates;
  `load_prompt` pins the content hash into every trace.
- `traceguard.sdk.tracer` — `@tracer.trace` decorator and `tracer.span()`
  context manager recording input hash, model/prompt versions, output, and
  perf into SQLAlchemy (SQLite by default).
- `traceguard.sdk.normalizer` — the single canonical `normalize_input` /
  `input_hash` (sorted keys, fixed float precision, normalized whitespace).
- `traceguard.sdk.wrappers.anthropic` — `wrap_anthropic` auto-instruments an
  Anthropic SDK client (extra: `traceguard[anthropic]`).
- `traceguard.sdk.wrappers.openai` — `wrap_openai` auto-instruments an OpenAI
  SDK client's `chat.completions` and `responses` calls (extra:
  `traceguard[openai]`).
- `traceguard.registry.replay` — curated, lockable replay sets
  (`create_replay_set` / `add_replay_item` / `lock_replay_set` /
  `build_locked_replay_set`); once locked, the store physically rejects any
  mutation (invariant 4).
- `traceguard.validators.lookahead` — invariant validators
  (`validate_feature_as_of`, `validate_model_timing`,
  `validate_reference_timing`, `assert_replay_set_locked`) that raise
  `InvariantViolation`; call them in pytest/CI to enforce all four look-ahead
  invariants.

All of the above are also re-exported from the top level (`from traceguard
import select_model, tracer, assert_replay_set_locked, ...`); the deep paths
stay valid as aliases. The package ships `py.typed`, so the annotations
(including the `Literal`-typed `select_model(..., strict=...)`) reach
type-checkers. Persistence is **fail-open**: a tracing/DB failure never breaks
or masks the instrumented call (set `TRACEGUARD_STRICT_PERSISTENCE=1` to fail
closed).

## Install

```bash
pip install traceguard
```

Requires Python 3.11+. Optional extras:
`pip install "traceguard[anthropic]"` / `pip install "traceguard[openai]"`
(Anthropic / OpenAI client wrappers) and
`pip install "traceguard[otel]"` (OpenTelemetry / OpenInference export to
Langfuse, Phoenix, or any OTLP backend).

## Example

```python
from datetime import datetime, timezone
from traceguard.registry.models import register_model, select_model
from traceguard.store.models import make_engine

engine = make_engine("sqlite:///traceguard.db")

register_model("demo-llm-2024", model_family="internal-ml",
               capability_class="general-llm",
               released_at=datetime(2024, 1, 10, tzinfo=timezone.utc),
               available_to_us_at=datetime(2024, 2, 1, tzinfo=timezone.utc),
               engine=engine)

# Backtesting as of mid-2025: models that arrived later are invisible.
model_id = select_model("general-llm",
                        available_at=datetime(2025, 6, 30, tzinfo=timezone.utc),
                        strict=True, engine=engine)
```

A complete runnable tour (synthetic data, no API keys) lives in
[examples/quickstart](https://github.com/lizhuojunx86/traceguard/tree/main/examples/quickstart).

## Contract

The binding interface contract — table schemas, SDK signatures, the four
look-ahead invariants, SemVer rules — is in
[docs/SPEC.md](https://github.com/lizhuojunx86/traceguard/blob/main/docs/SPEC.md).

Implemented: tracer, model/prompt registries, normalizer, all four look-ahead
invariants (1–4, including locked replay sets), Anthropic + OpenAI wrappers,
and a frozen public API. Not yet (post-1.0): drift checks + alerts, the full
replay executor / A-B compare tooling, a CLI, Postgres/TimescaleDB, Voyage
wrapper — see
[TRACEGUARD_ROADMAP.md](https://github.com/lizhuojunx86/traceguard/blob/main/TRACEGUARD_ROADMAP.md).

## Development

```bash
cd packages/traceguard
uv sync
uv run pytest        # 181 tests (4 skip without the contamination-hf extra)
```

## License

Apache-2.0.
