Metadata-Version: 2.4
Name: mod-trace
Version: 0.5.3
Classifier: Programming Language :: Rust
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Environment :: Console
Classifier: Operating System :: OS Independent
Classifier: Topic :: Software Development :: Build Tools
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
License-File: LICENSE
Summary: Rust CLI for inspecting ML model artifacts without loading the framework
Keywords: cli,ml,onnx,catboost,model,inspector,ci,diff
Requires-Python: >=3.9
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM

# mod-trace

Inspect ML model artifacts without loading the framework.

mod-trace is a small Rust CLI for answering a practical question:

```text
What is inside this model file?
```

It can inspect real artifacts such as CatBoost `.cbm` files, LightGBM `.txt`/`.lgb` text models, ONNX `.onnx` graphs, and PyTorch `.pt`/`.pth` checkpoints, then report structure, size, parameters, operator mix, rough inference cost, and changes between versions. All formats are read natively — no Python, framework, or runtime needed (CatBoost `--deep` is the one optional exception). The PyTorch reader is static: it sizes/names tensors and fingerprints weights without decoding exact shapes.

The most useful command is `explain-diff`, which says in plain English what changed between two model versions:

```sh
mod-trace explain-diff old_model.onnx new_model.onnx
```

(A secondary "tensor lab" for handcrafted JSON plans lives in [docs/tensor-lab.md](docs/tensor-lab.md).)

## Install

mod-trace is a single self-contained binary (written in Rust, published to PyPI as a wheel). The core commands need **no Python and no ML framework** at runtime.

```sh
pipx install mod-trace      # recommended: isolated, puts `mod-trace` on your PATH
# or
pip install mod-trace

mod-trace doctor            # shows which formats and helpers are available
```

- Prebuilt wheels for **Apple Silicon macOS**, **Linux x86_64**, and **Windows x64** — `pip` just downloads the binary, no build step.
- **Intel macOS** has no prebuilt wheel yet, so `pip install` there builds from source (needs a Rust toolchain).
- Optional: CatBoost `--deep` (decoded tree/leaf diffs) shells out to Python `catboost`; point `MODELLENS_PYTHON` at a Python that has it if needed. Everything else — including all of LightGBM, ONNX, and PyTorch — needs nothing extra.

## Core Commands

```sh
mod-trace doctor                                  # what formats/helpers are available
mod-trace inspect model.cbm                       # one model (also .lgb, .onnx, .pt)
mod-trace inspect --json model.onnx               # machine-readable
mod-trace inspect --deep model.cbm                # CatBoost: decoded splits (needs python catboost)
mod-trace diff old.cbm new.cbm                    # compare two versions
mod-trace explain-diff old.onnx new.onnx          # plain-English what changed
mod-trace check --max-size-growth 20% --fail-on-feature-change old.lgb new.lgb   # CI gate (exit != 0 = fail)
mod-trace convert model.pkl                       # un-pickle a model -> native (needs python)
```

Works on **CatBoost `.cbm`**, **LightGBM `.txt`/`.lgb`**, **ONNX `.onnx`**, and **PyTorch `.pt`/`.pth`/`.bin`**.

## Why This Exists

Data and ML engineers often inherit model artifacts:

- `fraud_model.cbm`
- `ranking_model.onnx`
- `model_v17.cbm`
- `candidate_model.onnx`

Before running them, it is useful to know:

- what type of model it is
- how large it is
- how many trees, parameters, nodes, or operators it contains
- what the rough per-row or per-forward-pass cost looks like
- what changed between two versions

mod-trace is not a model runtime. It is an artifact inspector.

## Doctor

`mod-trace doctor` lists which inspectors and optional helpers are available (each built-in format, the Python/`catboost` helper for `--deep`/`convert`, and which commands are usable). Add `--json` for CI/setup scripts to check availability programmatically.

## JSON Output

Use JSON when mod-trace is part of CI, release checks, or model registry automation:

```sh
mod-trace inspect --json path/to/model.cbm
mod-trace inspect --json path/to/model.onnx
mod-trace diff --json path/to/old_model.cbm path/to/new_model.cbm
mod-trace diff --json path/to/old_model.onnx path/to/new_model.onnx
```

The JSON diff is designed for checks such as:

- fail if file size, parameter memory, or estimated ops grows too much
- fail if CatBoost feature names, training config, or learned-state fingerprint changes unexpectedly
- fail if ONNX operator counts or initializer tensors change

`--deep` CatBoost reports are text-only for now because they are diagnostic dumps from CatBoost's native Python parser.

## CI Checks

Use `check` when a model artifact should fail promotion if it changes too much:

```sh
mod-trace check path/to/old_model.cbm path/to/new_model.cbm \
  --max-size-growth 20% \
  --fail-on-feature-change \
  --fail-on-training-config-change

mod-trace check path/to/old_model.onnx path/to/new_model.onnx \
  --max-size-growth 20% \
  --max-ops-growth 25% \
  --max-parameter-growth 30% \
  --fail-on-new-op
```

Check rules:

| Rule | Applies to | Fails when |
|------|------------|-----------|
| `--max-size-growth <pct>` | all | file size grows more than `<pct>` |
| `--max-parameter-growth <pct>` | ONNX | parameter count grows more than `<pct>` |
| `--max-ops-growth <pct>` | ONNX | estimated op count grows more than `<pct>` |
| `--fail-on-new-op` | ONNX | a new operator type appears |
| `--fail-on-feature-change` | CatBoost, LightGBM | feature names change |
| `--fail-on-training-config-change` | CatBoost, LightGBM | objective/learning rate/etc. change |

`check` prints a short PASS/FAIL report and exits nonzero when a rule fails. Any number of rules can be combined; any one failing fails the whole check.

## Explain Diff

`explain-diff` is the plain-English version of `diff` — it reports what actually changed between two model versions, not just raw numbers:

```sh
mod-trace explain-diff old_model.onnx new_model.onnx
```

```text
Model Change Explanation
------------------------
Type: ONNX
Old: old_model.onnx
New: new_model.onnx

Architecture:
  Attention layers:  12 -> 24
  Hidden size:       768 -> 1024
  Parameters:        110.0M -> 220.0M (+100%)
  Nodes:             420 -> 820 (+95%)

Estimated inference cost (static op proxy): +94%

New operators introduced:
  LayerNormalization

Summary:
  Grew from ~12 to ~24 attention layers; parameters +100%, estimated cost +94%.
```

Works for ONNX, CatBoost, and LightGBM (tree models report trees / leaves / learned-state instead of attention layers).

## Supported formats

| Format | Extensions | Commands | Native (no framework)? |
|--------|-----------|----------|------------------------|
| CatBoost | `.cbm` | inspect, diff, check, explain | yes (`--deep` uses Python catboost) |
| LightGBM | `.txt`, `.lgb` | inspect, diff, check, explain-diff | yes |
| ONNX | `.onnx` | inspect, diff, check, explain, explain-diff | yes |
| PyTorch | `.pt`, `.pth`, `.bin`, `.ckpt` | inspect, diff, explain-diff, check | yes (no torch) |

Per-format commands and **full example output** live next to the sample models:

- CatBoost → [examples/catboost/README.md](examples/catboost/README.md) (incl. `--deep`)
- LightGBM → [examples/lightgbm/README.md](examples/lightgbm/README.md)
- ONNX → [examples/onnx/README.md](examples/onnx/README.md)
- PyTorch → [examples/pytorch/README.md](examples/pytorch/README.md)

### Notes per format

- **CatBoost `--deep`** is optional and needs Python `catboost`; it decodes exact splits, leaf values, feature typing, and float borders. Plain `inspect`/`diff` need nothing.
- **PyTorch** is read statically from the `torch.save` zip (param counts, names, dtype, sampled weight fingerprint; ZIP64-safe). Shapes are not decoded. Legacy-pickle `.pt`/`.bin` give names-only; `.safetensors` is not supported (export to ONNX instead).
- **PyTorch → ONNX:** for graph-level detail or a `.safetensors`/legacy file, `torch.onnx.export(model, dummy, "model.onnx", opset_version=18)` then `mod-trace inspect model.onnx`. See [examples/pytorch/README.md](examples/pytorch/README.md).

## Convert (pickled / MLflow models)

mod-trace reads *native* model formats, but models are often stored as a **pickle** (`model.pkl`, e.g. logged by MLflow) — which is Python-only and can't be read without the framework. `convert` does the one-time translation:

```sh
mod-trace convert model.pkl                 # -> model.txt (LightGBM) or model.cbm (CatBoost)
mod-trace convert ./mlflow-model-dir --out /tmp/v3.txt
mod-trace inspect model.txt                 # now readable natively
```

This is the **one command that needs Python** (it loads the pickle with `lightgbm`/`catboost`, then re-saves natively). Point `MODELLENS_PYTHON` at an interpreter that has the model's library. Supported: LightGBM → `.txt`, CatBoost → `.cbm`. After converting, everything else is framework-free.

## Tensor Lab

A secondary lab for explaining transformer internals on small handcrafted JSON
plans (`trace`, `compare`, `why`, `validate`, `quiz`, `demo`). It is not the
primary product surface. See [docs/tensor-lab.md](docs/tensor-lab.md).

## What It Does Not Do

mod-trace does not:

- run inference
- train models
- load PyTorch directly
- require CatBoost, PyTorch, or ONNX Runtime for inspection
- provide GPU kernels
- replace framework-native debugging tools

## Privacy

Do not commit real model weights or private business artifacts. The repository ignores:

- `models/`
- `examples/*.onnx`
- `examples/*.cbm`

Use `examples/make_sample_catboost.py` when you need a shareable `.cbm` demo.

## Architecture

mod-trace has small, native inspection paths per format:

- CatBoost `.cbm` metadata scanner
- LightGBM `.txt`/`.lgb` text parser
- ONNX protobuf graph scanner
- PyTorch `.pt`/`.pth` zip/pickle scanner
- JSON tensor-plan analyzer

