Metadata-Version: 2.4
Name: dag-ml
Version: 0.3.37
Classifier: Development Status :: 4 - Beta
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Rust
Classifier: Typing :: Typed
Requires-Dist: numpy>=1.24 ; extra == 'multimodal'
Requires-Dist: nirs4all-methods>=1.3.2 ; extra == 'multimodal'
Requires-Dist: scikit-learn>=1.3 ; extra == 'multimodal'
Provides-Extra: multimodal
License-File: LICENSE
Summary: Python bindings for DAG-ML JSON contracts.
Home-Page: https://github.com/GBeurier/dag-ml
Author-email: "G. Beurier" <beurier@cirad.fr>
License-Expression: CECILL-2.1 OR AGPL-3.0-or-later
Requires-Python: >=3.11
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM

# dag-ml Python bindings

Thin PyO3/maturin bindings for DAG-ML JSON contracts.

This package validates, compiles and plans serialized DAG-ML contracts. Its
owning training entry point also executes the native DAG-ML coordinator while
operator implementations remain Python callbacks; no numerical or fold logic
is reimplemented in the binding.

## Build

This crate is excluded from the root cargo workspace (its `abi3-py311` floor
would force a Python >= 3.11 host on `cargo test --workspace` / `cargo
llvm-cov`), so build and test it through its own manifest against a Python
>= 3.11 interpreter:

```bash
PYO3_PYTHON=python3.11 cargo test --manifest-path crates/dag-ml-py/Cargo.toml
maturin build --release --features extension-module   # run from this crate dir
python3 ../../scripts/smoke_python_bindings.py        # after installing the wheel
PYTHONPATH=python python3.11 -m unittest discover -s tests
```

The source package also contains the tracked `_dag_ml.abi3.so` used by direct
`PYTHONPATH=crates/dag-ml-py/python` imports. After changing compiled Rust
inputs, refresh it with `maturin develop --release` inside an active Python
3.11+ virtual environment, run `python scripts/check_so_freshness.py`, and
smoke the public source-tree import. The freshness gate rejects dirty or
untracked Rust inputs when that tracked extension is unchanged.

## Python Surface

`dag_ml._dag_ml.probe_data_view_in_process(envelope_json, request_json,
view_callback)` exercises the host view callback without fitting a model. It
creates native parent/view handles, rejects IDs outside the envelope relations,
and requires a receipt with the same handle, view key, ordered IDs, and SHA-256
schema/content fingerprints. The caller supplies the request; this diagnostic
does not prove scheduler fold membership or that training consumed those
buffers. CV/refit and explicit phase execution accept their own view callbacks;
the host must bind each receipt to the actual buffers consumed by that task.
The standalone probe does not qualify those complete execution paths.

For a concrete host-managed pipeline without cross-validation, use
`execute_phase_in_process(dsl, envelope, controllers, callback, "REFIT",
training_sample_ids=[...])`. Rust verifies the supplied row ordering is an
exact unique permutation of the attested training identities and records it
in the effective campaign. Split invocations and unresolved operator choices
are refused; use the CV/refit entry point for those campaigns.

The same entry point with `phase="PREDICT"` requires a separately attested V2
prediction cohort and forbids `training_sample_ids`. It executes only PREDICT,
never fitting or selecting a model. Host-managed fitted state stays the host's
responsibility and is not implicitly promoted to a portable predictor package.
An optional `view_callback` receives every scheduler-selected request and its
opaque handle in REFIT or PREDICT. The host materializes its actual buffer and
returns a `DataViewReceipt`; Rust checks the handle, key, ordered identities and
canonical digests before dispatching the operator. This reuses the CV bridge
while retaining the explicit phase provider's training-order and cohort guards.
Omitting the callback preserves the existing handle-only path. Neither path
moves feature buffers into Rust or makes a host-model portability claim.

Both paths return JSON with native `node_results`, `scores`, `phase` and
`effective_plan`; absent target observations produce no invented score.

```python
import dag_ml

dag_ml.validate_graph_json(graph_json)  # raw JSON helper remains available

dsl = dag_ml.PipelineDslSpec(dsl_json)
controllers = dag_ml.ControllerManifests(controller_manifests_json)
artifact = dag_ml.compile_pipeline_dsl_artifact(dsl)
plan = dag_ml.build_execution_plan(
    "plan:example",
    artifact.graph,
    artifact.campaign_template,
    controllers,
)
plan_json = plan.json()

validated_request = dag_ml.TrainingRequest.from_path(
    "examples/fixtures/training/training_request_active_influence.v1.json"
)
validated_request = dag_ml.sign_training_request(unsigned_training_request)
relation_fingerprint = dag_ml.sample_relation_set_fingerprint_json(relations_json)
training_projection = validated_request.project()
package = dag_ml.PortablePredictorPackage.from_path(
    "examples/fixtures/training/portable_predictor_package.v1.json"
)

result = dag_ml.execute_training(
    native_training_request,
    data_envelopes={"model:base.x": signed_envelope},
    relations=sample_relations,
    training_influence=signed_influence,
    op_callback=run_node,
    outcome_id="outcome:example",
    run_id="run:example",
    bundle_id="bundle:example",
)
bundle = result.execution_bundle
scores = result.score_set
portable_artifacts = result.artifacts
result.detach()  # explicitly release callbacks, views and artifact handles
```

Portable package replay is available without the original `TrainingResult`.
Hosts pass the signed package, a `TrainingReplayRequest` whose `phase` is either
`PREDICT` or `EXPLAIN`, the current cohort data envelopes, and explicit sidecar
artifact handles:

```python
outcome = dag_ml.replay_loaded_predictor_package(
    package,
    replay_request,  # {"phase": "PREDICT"} or {"phase": "EXPLAIN"}
    data_envelopes,
    artifact_handles,
    run_node,
    outcome_id="outcome:package.replay",
    run_id="run:package.replay",
)
```

`PREDICT` must reproduce the requested package output bindings exactly.
`EXPLAIN` must emit at least one explanation block and may include the final
bound predictions for the requested bindings. The package remains handle-free;
all process-local model handles are supplied through `artifact_handles`.

`TrainingRequest`, `TrainingContractProjection`, `ParameterProjection`,
`CacheNamespace` and `PortablePredictorPackage` are validated by the native
`dag-ml-core` contracts. `sign_training_request()` canonicalizes and signs an
unsigned request through the same native structs, and
`sample_relation_set_fingerprint_json()` exposes the core relation fingerprint
for host-side data envelope assembly. `project_training_request()` is also
available as a functional facade. The binding does not reproduce parameter projection,
capability-derived influence or portability rules in Python.

`execute_training()` requires an envelope map keyed by the exact
`node_id.input_name` requirement key plus the matching relations and influence
manifest. The PyO3 layer releases the GIL while the core runs and reacquires it
only for controller callbacks. `TrainingResult` retains the controller registry,
attested provider and artifact store until `detach()` or object destruction;
portable outcome, bundle, scores, outputs and artifact metadata remain readable
after detach, while process-local handles are never serialized.

All Rust-side validation failures are raised as `dag_ml.DagMlError`. Native
errors expose `category`, `code`, `severity`, `remediation_hint`, `context`,
`context_json` and `descriptor_json` attributes for ADR-11-compatible handling.

