Metadata-Version: 2.4
Name: triggerflow
Version: 0.5.0
Summary: Utilities for ML models targeting hardware triggers
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: cookiecutter>=2.3
Requires-Dist: PyYAML>=6
Requires-Dist: kedro==1.0.0
Requires-Dist: kedro-datasets
Requires-Dist: kedro-mlflow==2.0.1
Requires-Dist: alkaid>=0.8.2
Requires-Dist: keras>=3
Requires-Dist: numpy>=1.24
Requires-Dist: pandas
Requires-Dist: pyarrow
Requires-Dist: awkward<3,>=2.8
Requires-Dist: dask==2025.3.0
Requires-Dist: distributed==2025.3.0
Requires-Dist: coffea>=2025.12
Provides-Extra: train
Requires-Dist: tensorflow; extra == "train"
Requires-Dist: scikit-learn; extra == "train"
Requires-Dist: matplotlib; extra == "train"
Requires-Dist: mplhep>=0.3; extra == "train"
Requires-Dist: uproot>=5; extra == "train"
Requires-Dist: hgq2>=0.2.0; extra == "train"
Provides-Extra: hls4ml
Requires-Dist: hls4ml; extra == "hls4ml"
Provides-Extra: conifer
Requires-Dist: xgboost; extra == "conifer"
Requires-Dist: conifer; extra == "conifer"
Provides-Extra: dev
Requires-Dist: pytest-cov~=3.0; extra == "dev"
Requires-Dist: pytest-mock<2.0,>=1.7.1; extra == "dev"
Requires-Dist: pytest~=7.2; extra == "dev"
Requires-Dist: ruff~=0.1.8; extra == "dev"

# Machine Learning for Hardware Triggers

[![Documentation](https://img.shields.io/badge/docs-CERN-blue.svg)](https://l1t-calibrations-mlops.docs.cern.ch/triggerflow/)
[![PyPI version](https://badge.fury.io/py/triggerflow.svg)](https://badge.fury.io/py/triggerflow)

`triggerflow` takes a trained Keras model plus a numpy preprocessing function and
exports them as a single [alkaid](https://pypi.org/project/alkaid/) HLS kernel
for FPGA deployment in the CMS Level-1 trigger:

- one traced graph: input quantization, preprocessing and the network are all
  part of the exported kernel, so training and firmware evaluate the same numbers,
- a CMSSW-ready artifact: `emulator/<name>/<name>.so` (hls4mlEmulator ABI)
  inside a self-describing TriggerModel workspace whose `metadata.json` records
  the fixed object/feature order the producer uses to assemble the model input,
- parity tooling: software vs. firmware predictions event by event, from Python
  or from a CMSSW dump.

`TriggerModel` also supports the `hls4ml` (Keras) and `conifer` (XGBoost)
compiler backends - alkaid is the one with the full e2e CMSSW deployment path.
MLflow utilities are shipped for logging, versioning and loading of trigger
models.

## Installation

```bash
pip install triggerflow
# training extras (tensorflow, scikit-learn, matplotlib):
pip install triggerflow[train]
# compiler backends next to alkaid:
pip install triggerflow[hls4ml]   # hls4ml (Keras)
pip install triggerflow[conifer]  # conifer (XGBoost)
```

## Usage

```python
import numpy as np
from triggerflow.core import TriggerModel
from my_project.preprocessing import preprocess   # standard numpy ops

trigger_model = TriggerModel(
    config="triggermodel_config.yaml",   # compiler + deployment spec
    native_model=model,                  # trained Keras model
    preprocess=preprocess,               # traced into the kernel
)
trigger_model()                          # export workspace with the .so (needs make/g++)

output_software = trigger_model.software_predict(x_preprocessed)
output_firmware = trigger_model.firmware_predict(x_raw)   # raw ABI inputs

trigger_model.save("triggermodel.tar.xz")

# in a separate session:
from triggerflow.core import TriggerModel
reloaded = TriggerModel.load("triggermodel.tar.xz", overwrite=True)
output_firmware = reloaded.firmware_predict(x_raw)
```

## The model config

```yaml
compiler:
  name: "my_model"
  ml_backend: "keras"
  compiler: "alkaid"
  n_outputs: 1
  clock_period: 5.0
  # ABI quantization of the graph as [sign, integer, fraction] bits:
  # [1, 22, 0] -> ap_fixed<23,23> raw GT hardware integers
  in_kif: [1, 22, 0]
  out_kif: [1, 6, 10]
  weight_frac_bits: 10          # snap weights to a 2**-10 grid
  input_shape: [14]             # raw feature count (preprocessed width is implicit)

# Single source of truth for the fixed object/feature order - stored in the
# workspace metadata.json at export; the CMSSW producer cfi mirrors it.
deployment:
  inputs:
    - tag: [l1tGTProducer, CL2JetsSC4]
      n_objects: 4
      features: [pT, eta, phi]
      sort: "pT:desc"
    - tag: [l1tGTProducer, CL2EtSum]
      n_objects: 1
      features: [pT, phi]
  decision:
    threshold: 0.0
    pass_above: true
    trigger_types: 1
```

Quantization and precision are not configured behind your back: the values above
become `quantize()` calls inside the traced graph. Preprocessing rules that keep
the traced bit widths small: keep constants dyadic (or quantize right after
them) and quantize before every LUT op.

## Logging with MLflow

```python
import mlflow
from triggerflow.mlflow_wrapper import log_model

mlflow.set_tracking_uri("https://ngt.cern.ch/models")   # or omit for ./mlruns
experiment_id = mlflow.create_experiment("example-experiment")

with mlflow.start_run(run_name="trial-v1", experiment_id=experiment_id):
    log_model(trigger_model, registered_model_name="TriggerModel")
```

`log_model` archives the TriggerModel, logs it as a PyFunc model and registers
it in the Model Registry; see the
[MLflow docs](https://l1t-calibrations-mlops.docs.cern.ch/triggerflow/mlflow/)
for loading models back and CI integration (`setup_mlflow`).

## Kedro starter

The package ships a cookiecutter starter with a minimal working Alkaid example
(5 pipelines: `load_data -> data_processing -> model_training ->
model_validation -> compile`, bundled test data):

```bash
triggerflow new demo_pipeline
cd demo_pipeline
conda env update -n demo_pipeline --file environment.yml --prune
conda activate demo_pipeline
kedro run
```

The `compile` pipeline writes `data/06_compile/` (predictions, `model_aucs.json`,
the `triggermodel.tar.xz` archive) and `triggermodel/` (the workspace: `.so` +
`metadata.json`). Its pipelines are tracked with MLflow
(hyperparameters, metrics, and a Model Registry version of the exported
TriggerModel); by default everything goes to the project-local `./mlruns`
store - set `MLFLOW_TRACKING_URI` to track on a server instead.

For a full CMSSW deployment round trip (L1 ntuples, `L1GTScoreProducer`, bit-exact
parity) see `examples/example_pipeline/`.

## Development

```bash
pip install -e .[dev,train]
pytest tests/test_core.py                      # fast unit tests
pytest tests                    # full suite (includes the starter e2e)
ruff check src tests
```

### Note: this package does not install a training environment; projects declare
their own dependencies via the starter's `environment.yml`.
