Metadata-Version: 2.4
Name: midas-pipeline
Version: 0.2.1
Summary: Unified MIDAS HEDM orchestrator. FF is the single-scan degeneracy of PF; one orchestrator, --scan-mode {ff,pf}.
Author-email: Hemant Sharma <hsharma@anl.gov>
License-Expression: BSD-3-Clause
Project-URL: Homepage, https://github.com/marinerhemant/MIDAS
Project-URL: Documentation, https://github.com/marinerhemant/MIDAS
Project-URL: Issues, https://github.com/marinerhemant/MIDAS/issues
Keywords: MIDAS,HEDM,FF-HEDM,PF-HEDM,scanning,diffraction,pipeline,PyTorch,multi-detector,crystallography,polycrystal,tomography
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Operating System :: OS Independent
Classifier: Topic :: Scientific/Engineering :: Physics
Requires-Python: >=3.9
Description-Content-Type: text/markdown
Requires-Dist: numpy>=1.22
Requires-Dist: torch>=2.0
Requires-Dist: zarr<3,>=2.14
Requires-Dist: h5py>=3.7
Requires-Dist: tifffile
Requires-Dist: pandas>=1.5
Requires-Dist: fabio>=0.14
Requires-Dist: midas-zipper>=0.1.0
Requires-Dist: midas-hkls[cif]>=0.2.0
Requires-Dist: midas-peakfit>=0.2.0
Requires-Dist: midas-transforms>=0.1.1
Requires-Dist: midas-index>=0.3.0
Requires-Dist: midas-fit-grain>=0.1.0a0
Requires-Dist: midas-process-grains>=0.1.0
Requires-Dist: midas-diffract>=0.1.0
Requires-Dist: midas-parsl-configs>=0.1.0
Requires-Dist: midas-stress>=0.6.0
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: pytest-cov; extra == "dev"
Requires-Dist: matplotlib; extra == "dev"
Requires-Dist: ipykernel; extra == "dev"
Requires-Dist: midas-hkls[cif]>=0.2.0; extra == "dev"
Provides-Extra: notebook
Requires-Dist: matplotlib; extra == "notebook"
Requires-Dist: ipykernel; extra == "notebook"
Requires-Dist: jupyter; extra == "notebook"
Provides-Extra: fast
Requires-Dist: numba>=0.65; extra == "fast"

# midas-pipeline

End-to-end MIDAS HEDM orchestrator. **FF is the single-scan degeneracy of PF.** One package, one CLI, two scan modes.

## Status

**0.1.0 — end-to-end PF and FF paths live.** The scanning indexer matches the C `IndexerScanningOMP` reference on its 1-voxel C-parity gate (seed identity, solution counts, voxel-center positions exact; orientation matrices within mrad-scale, the refiner closes the gap downstream). Real-data validation: Wenxi CP-Ti consolidation_pf reproduces the legacy `pf_MIDAS.py` grain count (770 == 770, all common). Park22 P5c parity gate now runs in ~6.7s vs the original 790s after the scanning-indexer position-grid fix.

Stages call in-process Python kernels via `midas-index` / `midas-fit-grain` / `midas-transforms` / `midas-stress`. FF mode shells out to `python -m midas_index` and `python -m midas_fit_grain` (same kernels, subprocess for the FF parity-preserving pattern). No CUDA C; GPU is torch-only.

## Install

```bash
pip install -e packages/midas_pipeline
```

## CLI

```bash
midas-pipeline run --scan-mode {ff,pf,auto} --params Parameters.txt --result rundir/
midas-pipeline status rundir/
midas-pipeline resume rundir/ --from <stage>
midas-pipeline reprocess rundir/
midas-pipeline inspect rundir/LayerNr_1/
midas-pipeline simulate --out simdir/ --params Parameters.txt
midas-pipeline seed --params ... --output UniqueOrientations.csv
```

When `--scan-mode` is omitted (default `auto`), the CLI sniffs the parameter file: `nScans > 1` or presence of `BeamSize` / scanning keys → `pf`, otherwise `ff`. For PF mode, `--n-scans`, `--scan-step`, `--beam-size`, and `--scan-pos-tol` default to values in the params file (CLI flags override).

### Indexer backend

```bash
midas-pipeline run --indexer-backend {python,c-omp} ...
```

`python` (default) — in-process numba/torch indexer. Portable (CPU/CUDA/MPS), differentiable, slower on large PF datasets.

`c-omp` — bundled unified C binary (`midas_indexer`) from `midas-index`. Requires midas-index installed with a working OpenMP toolchain (macOS: `brew install libomp`). ~290× faster than the Python path on real PF datasets (per the Wenxi CP-Ti benchmark in `packages/midas_index/dev/`). Output is bit-identical to the Python path on the PF parity gate.

## Coexistence with `midas-ff-pipeline`

The legacy `midas-ff-pipeline` console-script is preserved as an independent FF orchestrator (its own kernels, its own CLI). It is **not** deprecated by `midas-pipeline run --scan-mode ff` — both paths invoke the same `midas-index` / `midas-fit-grain` kernels under the hood, and both stay green on the FF parity gate. Pick whichever is more convenient for your workflow.

## Architecture

- **One orchestrator** with a mode-dependent `STAGE_ORDER`.
- **Shared kernel packages** (`midas-index`, `midas-fit-grain`, `midas-transforms`, etc.) extended in place; FF behavior preserved by parity gates.
- **PF-only modules** live inside `midas_pipeline` (`find_grains/`, `sinogen`, `recon/`, `fuse`, `potts`, `em_refine`, `seeding/`).
- **Differentiability + multi-device** mandatory on every new compute path (CPU / CUDA / MPS via torch).

## Constraints

- No CUDA C; GPU support is torch-only.
- No deletions of legacy code in this effort.
- `midas-process-grains` is FF-only; PF consolidation is fresh pure-Python.
- `utils/calcMiso.py` is not imported by this package; all orientation math comes from `midas-stress`.
- `TOMO/midas_tomo_python.py` is imported in place, not relocated.
