Metadata-Version: 2.4
Name: nmrmetaproc
Version: 1.0.22
Summary: NMR Metabolomics Spectral Processor - raw Bruker FID to analysis-ready CSV
Author-email: Folorunsho Bright Omage <omagefolorunsho@gmail.com>
License-Expression: MIT
Project-URL: Homepage, https://github.com/omagebright/nmrmetaproc
Project-URL: Repository, https://github.com/omagebright/nmrmetaproc
Keywords: NMR,metabolomics,spectral processing,chemometrics,Bruker
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
Classifier: Topic :: Scientific/Engineering :: Chemistry
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: NOTICE
License-File: AUTHORS.md
Requires-Dist: nmrglue>=0.9
Requires-Dist: numpy>=1.21
Requires-Dist: scipy>=1.7
Requires-Dist: pandas>=1.3
Requires-Dist: tqdm>=4.60
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0; extra == "dev"
Provides-Extra: gui
Requires-Dist: PyQt6>=6.5; extra == "gui"
Requires-Dist: pyqtgraph>=0.13; extra == "gui"
Requires-Dist: PyYAML>=6.0; extra == "gui"
Provides-Extra: benchmark
Requires-Dist: rdata>=1.0; extra == "benchmark"
Dynamic: license-file

# nmrmetaproc

[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Python](https://img.shields.io/badge/python-3.9%2B-blue)](https://www.python.org/)
[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.19194107.svg)](https://doi.org/10.5281/zenodo.19194107)

**NMR Metabolomics Spectral Processor**

`nmrmetaproc` converts raw Bruker NMR FID files into clean, analysis-ready spectral matrices (CSV format) suitable for PCA, PLS-DA, pathway analysis, and other downstream metabolomics workflows. It implements a rigorous, reproducible processing pipeline with automatic phase correction, chemical-shift referencing, robust baseline correction, spectral alignment, and Probabilistic Quotient Normalization (PQN).

**Authors:** Folorunsho Bright Omage, Toyin Bright Omage, Ljubica Tasic  
**ORCID:** [0000-0002-9750-5034](https://orcid.org/0000-0002-9750-5034)  
**Email:** omagefolorunsho@gmail.com  
**Provenance:** [Zenodo DOI](https://doi.org/10.5281/zenodo.19194107), [CITATION.cff](CITATION.cff), [AUTHORS.md](AUTHORS.md), [NOTICE](NOTICE), [PROVENANCE.md](PROVENANCE.md)

---

## Features

- Reads raw Bruker FID files (`fid`/`*.fid` + `acqus`/`acqu`) directly, no conversion needed
- Automatically discovers valid Bruker spectra under a parent directory and splits mixed data into scientifically comparable batches by cohort and pulse program
- Applies pulse-program-specific and sample-context-aware processing presets for common Bruker 1D 1H experiments (`noesygppr1d` including `noesy1d`/`noesyd` aliases, `cpmgpr1d`, `zg30`, `ledbpgppr2s1d`)
- Full processing pipeline in correct order:
  1. Exponential apodization (line broadening)
  2. Zero-filling
  3. Fast Fourier Transform
  4. **Automatic phase correction** (ACME algorithm, no fixed phase values)
  5. Context-aware chemical-shift referencing to TSP/DSS, TMS, residual CHCl3, or lipid methylene as appropriate
  6. Asymmetric least-squares (ALS) baseline correction
  7. Negative-value handling with per-sample logging
  8. Solvent/context region exclusion, for example aqueous water or residual CHCl3
  9. Spectral alignment (whole-spectrum correlation shift or reference-peak)
  10. Configurable region exclusion
  11. Uniform binning
  12. **PQN normalization** (default), or total area, TSP reference, none
- Per-sample quality control: SNR, reference-peak linewidth, solvent-region score, clean pass/warn/fail separation, and optional adaptive second-pass audit
- Clean CSV outputs ready for MetaboAnalyst, R, MATLAB, machine learning, and chemometrics
- Works on Windows, macOS, and Linux

---

## Download

Pre-built desktop applications for Windows, macOS, and Linux are published to the [Releases page](https://github.com/omagebright/nmrmetaproc/releases/latest):

| Platform | File | How to use |
|----------|------|-----------|
| Windows 10 / 11 (x64) | `nmrmetaproc-gui-windows-x64.zip` | Unzip and run `nmrmetaproc-gui.exe` |
| macOS 12+ (Apple Silicon + Intel) | `nmrmetaproc-gui-macos.dmg` | Open the DMG, drag `nmrmetaproc.app` to Applications |
| Linux (glibc 2.31+, x86_64) | `nmrmetaproc-gui-linux-x86_64.AppImage` | `chmod +x` the file and double-click |

These bundles contain Python, all dependencies, and the GUI app. You do **not** need to install Python separately to use them.

If you prefer Python integration (CLI + scriptable API), install from PyPI instead -- see [Installation](#installation) below.

### First-launch notes

The desktop bundles are **unsigned** -- we do not yet hold an Apple Developer ID or a Windows EV code-signing certificate, so the operating system shows a warning the first time you open the app. The app itself is the same code published on PyPI; you are bypassing OS-level signature checks, not malware checks.

**macOS (Gatekeeper)** -- if you see *"Apple could not verify nmrmetaproc is free of malware"* with no Open button, do **one** of the following after dragging the .app to /Applications:

- **Terminal one-liner** (fastest):
  ```bash
  xattr -dr com.apple.quarantine /Applications/nmrmetaproc.app
  ```
  Then launch normally.

- **GUI path**: click **Done** on the dialog (do **not** click Move to Bin), then open **System Settings -> Privacy & Security**, scroll to the **Security** section, click **Open Anyway** next to the "nmrmetaproc was blocked" line, and re-launch the app.

**Windows (SmartScreen)** -- on first launch you may see *"Windows protected your PC"*. Click **More info**, then **Run anyway**.

**Linux (AppImage permissions)** -- the file needs the executable bit set:
```bash
chmod +x nmrmetaproc-gui-linux-x86_64.AppImage
```
If your distro complains about FUSE, install `libfuse2` (Ubuntu: `sudo apt install libfuse2`).

---
## Installation

```bash
pip install nmrmetaproc
```

Or from source:

```bash
git clone https://github.com/omagebright/nmrmetaproc.git
cd nmrmetaproc
pip install -e .
```

**Dependencies:** `nmrglue`, `numpy`, `scipy`, `pandas`, `tqdm`

---

---

## Try it Now - Interactive Demo

[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/omagebright/nmrmetaproc/blob/main/examples/colab_demo_real_data.ipynb)

**Try nmrmetaproc instantly** with real de-identified clinical NMR data from a thrombosis study! The interactive Google Colab notebook demonstrates the complete workflow from raw Bruker FID files to publication-quality figures and statistical analysis.

**Demo features:**
- Process 6 real NMR samples (3 Control + 3 Thrombosis) 
- 600 MHz Bruker AVANCE III data
- Quality control visualization
- Group comparison plots  
- Principal component analysis
- Zero installation required - runs in your browser!

[**Launch Demo**](https://colab.research.google.com/github/omagebright/nmrmetaproc/blob/main/examples/colab_demo_real_data.ipynb)

## Command-Line Usage

### Full Processing Pipeline

Point `nmrmetaproc` at the parent Bruker directory. By default it discovers all valid spectra, separates incompatible cohorts and pulse programs into independent batches, applies the registered pulse-program presets, and writes one analysis-ready matrix per batch.

```bash
nmrmetaproc process /path/to/bruker/data --output ./results
```

To run the transparent adaptive second pass from the CLI, add:

```bash
nmrmetaproc process /path/to/bruker/data --output ./results --adaptive-second-pass
```

The first-pass QC remains available as `qc_report_initial.csv`; the selected
second-pass candidate and before/after QC metrics are recorded in
`adaptive_qc_report.csv`. A corrected pass requires an improved rerun QC with
a measurable reference peak; candidates that lose reference evidence are rejected.

Batch output layout:

```text
results/
|-- batch_manifest.csv
|-- FIDs_Toyin__zg30/
|   |-- spectral_matrix.csv
|   |-- qc_report.csv
|   |-- qc_report_initial.csv              # when adaptive second pass is enabled
|   |-- adaptive_qc_report.csv             # when adaptive second pass is enabled
|   |-- acquisition_parameters.csv
|   `-- processing.log
`-- FIDs_Julia__cpmgpr1d/
    |-- spectral_matrix.csv
    |-- qc_report.csv
    |-- qc_report_initial.csv              # when adaptive second pass is enabled
    |-- adaptive_qc_report.csv             # when adaptive second pass is enabled
    |-- acquisition_parameters.csv
    `-- processing.log
```

Use `--single-matrix` only when the input directory is already one homogeneous cohort and pulse program:

```bash
nmrmetaproc process /path/to/one_batch --output ./results --single-matrix
```

### QC Scan Only

```bash
nmrmetaproc qc /path/to/data --output ./qc_results
```

### Inspect Available Samples

```bash
nmrmetaproc info /path/to/data
```

---

## Python API

```python
from nmrmetaproc import NMRProcessor

processor = NMRProcessor(
    lb=0.3,
    bin_width=0.01,
    normalization="pqn",
    ppm_range=(0.5, 9.5),
    snr_threshold=10.0,
    linewidth_threshold=2.5,
    align="correlation",
)

results = processor.process("/path/to/bruker/data")

print(results.spectral_matrix)   # rows=samples, columns=ppm bins
print(results.qc_report)         # SNR, linewidth, pass/fail per sample

results.save("./output")
```

---

## Output Files

| File | Description |
|------|-------------|
| `spectral_matrix.csv` | Rows = samples (passed QC), columns = ppm bin centres. PQN/total-area normalized; **not** scaled |
| `spectral_matrix_labeled.csv` | Optional. Labelled feature matrix (only when peak-label export is enabled). Solvent/reference labels marked as solvent are excluded |
| `spectral_matrix_analysis_ready.csv` | Optional. Low-information bins removed, robust per-bin outlier screen, **robust feature scaling already applied** (median-centred, MAD-scaled), so do **not** autoscale again in MetaboAnalyst/sklearn. Robust steps are skipped for cohorts with < 8 samples |
| `analysis_ready_report.csv` | Optional. Audit table for the raw-bin analysis-ready matrix |
| `spectral_matrix_labeled_analysis_ready.csv` | Optional. ML/chemometrics-ready version of the labelled matrix, written when both label export and analysis-ready export are enabled |
| `analysis_ready_labeled_report.csv` | Optional. Audit table for the labelled analysis-ready matrix |
| `qc_report.csv` | Final QC used for matrix inclusion. Corrected samples are marked with adaptive stage/status columns when adaptive QC is enabled |
| `qc_report_initial.csv` | Optional. Untouched first-pass QC before adaptive second-pass reprocessing |
| `adaptive_qc_report.csv` | Optional. Before/after second-pass audit table: trigger, candidate, score delta, SNR, linewidth, residual score, reference ppm/QC, warnings |
| `acquisition_parameters.csv` | Bruker metadata, inferred context/solvent, pulse program, effective per-sample processing settings, and adaptive candidate settings when applied |
| `processing.log` | Full processing log with all parameters and per-sample status |

---

## Data Format

Each sample must be in its own directory containing:
- `fid` or `*.fid` - binary FID data (interleaved real/imaginary int32)
- `acqus` or `acqu` - acquisition parameter file

```
data_root/
|-- sample_001/
|   |-- fid
|   `-- acqus
`-- sample_002/
    |-- fid
    `-- acqus
```

Nested layouts are also supported and discovered automatically.

---

## Reference and QC notes

The software chooses the reference from both pulse program and sample context. Aqueous biofluids use TSP/DSS at 0.000 ppm. Non-aqueous spectra with explicit TMS evidence use TMS at 0.000 ppm. CDCl3 lipid extracts without TMS evidence use residual CHCl3 at 7.260 ppm and exclude 7.24-7.28 ppm from the matrix. If `pdata/1/procs` or `pdata/1/proc` is present, NMRMetaProc uses Bruker `OFFSET`, `SW_p`, and `SF` to construct the calibrated ppm axis and records the source in `acquisition_parameters.csv`. Without Bruker processed-axis metadata, the software attempts a robust measured reference lock; if no lock is obtained, the sample fails QC rather than entering the matrix with an uncalibrated chemical-shift axis. WARN samples are accepted into the matrix but carry review items; clean PASS, WARN, and FAIL are reported separately in the GUI and QC report.

A public Bruker 1D FID fixture from the MMCD/nmrglue example mirror is included in the test suite to validate raw FID reading, PULPROG alias dispatch, ppm-axis direction, and dominant solvent-region placement.

For a biological metabolomics benchmark, `benchmarks/pepsnmr_human_serum/` provides a reproducible runner for the public Bioconductor PepsNMRData package. It downloads raw Bruker human-serum FIDs plus the package's published `FinalSpectra_HS` matrix, processes the raw FIDs with NMRMetaProc, and writes an auditable benchmark JSON. PepsNMRData does not provide human QC pass/fail labels, so the benchmark checks external discovery, pulse-program dispatch, finite matrix export, and transparent QC diagnostics rather than forcing artificial QC agreement.

An ST000020 urine pilot benchmark against the Metabolomics Workbench depositor matrix exposed a previous reference-axis defect. After honoring Bruker `procs` axis metadata, the rerun processed 12/12 FIDs with median residual shift 0.005 ppm, median per-sample Pearson 0.968, median cosine 0.972, and top-20 per-bin Pearson 0.984 after re-integration to the depositor bins. The Spearman-rank diagnostic remains lower because the depositor matrix uses manual preprocessing and variable-width intelligent bucketing, so it is treated as a diagnostic rather than a hidden pass/fail claim.

---

## Citing and Provenance

If you use `nmrmetaproc` in your research, please cite:

```
Omage, F. B., Omage, T. B., & Tasic, L. (2026). nmrmetaproc: NMR Metabolomics Spectral Processor (Version 1.0.22).
Zenodo. https://doi.org/10.5281/zenodo.19194107
```

The canonical public provenance records are the GitHub repository, annotated
release tags, GitHub Releases, [CITATION.cff](CITATION.cff), [AUTHORS.md](AUTHORS.md),
[NOTICE](NOTICE), [PROVENANCE.md](PROVENANCE.md), and the Zenodo DOI. The MIT
License permits reuse and redistribution only with the required copyright and
permission notices retained. Scientific use should preserve authorship and cite
the software DOI.

The PQN normalization method:

> Dieterle, F., Ross, A., Schlotterbeck, G., & Senn, H. (2006). Probabilistic quotient
> normalization as robust method to account for dilution of complex biological mixtures.
> *Analytical Chemistry*, 78(13), 4281-4290. https://doi.org/10.1021/ac051632c

---

## Development

```bash
git clone https://github.com/omagebright/nmrmetaproc.git
cd nmrmetaproc
pip install -e ".[dev]"
pytest tests/ -v
```

---

## License

MIT License. See [LICENSE](LICENSE) for details. Redistribution and derivative
work must retain the copyright and permission notices. See [NOTICE](NOTICE) and
[PROVENANCE.md](PROVENANCE.md) for authorship and citation records.


