Metadata-Version: 2.4
Name: pygecko-gc
Version: 0.1.0
Summary: An open-source Python library for the parsing, processing and analysis of GC/MS and GC/FID raw data
Author-email: Felix Katzenburg <felix.katzenburg@uni-muenster.de>, Florian Boser <florian.boser@uni-muenster.de>
License: MIT License
Project-URL: Homepage, https://github.com/FelixKatz77/pyGecko
Project-URL: Documentation, https://pygecko.readthedocs.io/en/latest/
Project-URL: Repository, https://github.com/FelixKatz77/pyGecko
Project-URL: Issues, https://github.com/FelixKatz77/pyGecko/issues
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: POSIX :: Linux
Classifier: Operating System :: Microsoft :: Windows :: Windows 11
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE.txt
Requires-Dist: brain_isotopic_distribution>=1.5.14
Requires-Dist: epam.indigo>=1.13.0
Requires-Dist: lxml>=6.1.0
Requires-Dist: matplotlib>=3.6.2
Requires-Dist: netCDF4>=1.6.5
Requires-Dist: numpy>=1.26.3
Requires-Dist: pandas>=2.0.3
Requires-Dist: pybaselines>=1.0.0
Requires-Dist: psims>=1.3.2
Requires-Dist: pymzml>=2.5.2
Requires-Dist: pyteomics[xml]>=4.6.2
Requires-Dist: rdkit>=2023.3.1
Requires-Dist: reportlab>=4.0.5
Requires-Dist: scikit-learn>=1.7.2
Requires-Dist: scipy>=1.11.4
Requires-Dist: statsmodels>=0.14.0
Requires-Dist: xarray>=2023.6.0
Provides-Extra: ord
Requires-Dist: ord-schema>=0.5.9; extra == "ord"
Provides-Extra: test
Requires-Dist: pytest>=8; extra == "test"
Requires-Dist: pytest-cov>=5; extra == "test"
Provides-Extra: docs
Requires-Dist: sphinx>=7; extra == "docs"
Requires-Dist: sphinx-rtd-theme>=2; extra == "docs"
Dynamic: license-file

# pyGecko
<img src="docs/pyGecko_icon.png" alt="pyGecko_Logo" width="300" height="300"/>

> pyGecko an open-source Python library for the parsing, processing and analysis of GC-MS and GC-FID raw data.

With increasing amounts of analytical and metadata generated in HTE, data processing and analysis quickly become a 
workflow's limiting step if conducted manually. The automated processing of analytical data opens up time for chemists 
to focus on relevant outcomes, enables the standardized storage of reaction data, and facilitates the integration of 
analytical methods into closed-loop systems. Herein we present pyGecko, an open-source Python library for the parsing,
processing and analysis of GC-MS and GC-FID raw data. pyGecko offers a variety of analysis tools for the automated or 
semi-automated handling of GC measurements and sequences. This includes the interpretation of measurements in the context 
of the experiment, the automatic identification of internal standards and compound identifications based on retention 
times, the mass of a molecular ion or fragment and spectral comparison. Quantification relative to an internal standard 
can be performed for GC-FID measurements. Results of an analysis as well as chromatograms and spectra can be visualized 
and reported in standardized formats like the Open Reaction Database (ORD) schema. pyGecko is designed to be easily 
integrated into automated workflows and can be used as a stand-alone tool or as a python library.

Preprint: https://chemrxiv.org/engage/chemrxiv/article-details/66adfc465101a2ffa8001761  <br>
Paper: https://doi.org/10.1039/D4DD00347K

## Installation

> [!IMPORTANT]
> To read vendor files you need to install the msConvert tool from ProteoWizard. You can download it from [here](http://proteowizard.sourceforge.net/download.html).
> You need to specify the path to the msConvert.exe before the first run of pyGecko.

pyGecko requires Python 3.10 or newer and is published on PyPI as `pygecko-gc`
(the import name stays `pygecko`):

```bash
pip install pygecko-gc
```

Optional extras: `pip install "pygecko-gc[ord]"` adds Open Reaction Database export
(`Reaction_Parser`), `"pygecko-gc[test]"` the test dependencies and `"pygecko-gc[docs]"` the
documentation build.

To work on pyGecko itself, install an editable checkout instead:

```bash
git clone https://github.com/FelixKatz77/pyGecko.git
cd pyGecko
pip install -e ".[test]"
```

To install the exact, pinned set of dependency versions instead of the newest compatible ones,
use [uv](https://docs.astral.sh/uv/) with the committed lock file:

```bash
uv sync
```

### Configuring msConvert

pyGecko looks for the msConvert executable each time a vendor file is converted, in this order:

1. the `PYGECKO_MSCONVERT` environment variable, set to the full path of the executable
2. an `msconvert` found on your `PATH`

Set the variable once for your user account, e.g. on Windows via *Start → "Edit environment
variables for your account" → New* (name `PYGECKO_MSCONVERT`, value
`C:\path\to\msconvert.exe`), or on Linux/macOS by adding
`export PYGECKO_MSCONVERT=/path/to/msconvert` to your shell profile. Inside a script or notebook
you can also set it for the current session before calling pyGecko:

```python
import os
os.environ["PYGECKO_MSCONVERT"] = r"C:\path\to\msconvert.exe"
```

Without msConvert, open formats (`.mzML`, `.mzXML`, `.cdf`, `.xy`, `.csv`) still work.


## Documentation
The documentation for pyGecko can be found [here](https://pygecko.readthedocs.io/en/latest/).

## Running the tests

```bash
pip install -e ".[test,ord]"
pytest
```

Two integration tests load Agilent `.D` directories and therefore need a configured msConvert
executable; they fail without one. To skip them, run `pytest -m "not msconvert"`.

The normal offline suite includes small, attributed `.xy` and mzML excerpts from the pyGecko study.
It also checks ORD/PDF export using the corresponding plate metadata. Run it with the coverage gate:

```bash
pytest -m "not msconvert and not slow" --cov=pygecko --cov-report=term-missing --cov-fail-under=80
```

### Full study-data regressions

The complete plate tests use release 1.2 of the study dataset, pinned to
[Zenodo record 14316687](https://zenodo.org/records/14316687). Raw archives and extracted files are
kept in the ignored `.test-data/` directory and are never added to Git. Download all three
checksum-verified archives, then run the plate regressions:

```bash
python -m tests.support.fetch_zenodo
PYGECKO_REAL_DATA_DIR="$PWD/.test-data/zenodo/14316687" \
  pytest tests/real_data -m "realdata and slow"
```

These regressions process all 96 FID and all 96 mzML injections for thiolation,
Buchwald–Hartwig, and AD-HoC. Thiolation and AD-HoC reproduce the checked-in yields, retention
times, and analyte assignments exactly. Buchwald–Hartwig reproduces every assignment and retention
time; 26 of its 27 reported yields are exact. Well C9 is expected to be 67% with the current peak
overlap-border correction rather than the paper's 74%, and the test requires its `overlap` flag.
The same full run is scheduled weekly in CI and can be started with `workflow_dispatch`.

## Usage
For non-automated workflows pyGecko is best used with jupyter notebooks. The notebooks folder of the repository contains
examples for the usage of pyGecko for the quantitative analysis of reaction outcomes and spectral matching. The Python 
scripts used to perform the data processing for the publication can be found in the examples folder. GC-MS and GC-FID 
raw data for all experiments is available on Zenodo.

### Split-GC: single-injection FID + MS

For instruments that split one GC column post-column to both an MS and a Polyarc-FID detector, both traces
come from a single injection and share a retention-time axis. `SplitGC_Parser.load_sequence` reads such an
OpenLab `.rslt`/`.sirslt` folder and returns paired FID/MS sequences. Because the detectors share a time axis,
FID and MS peaks can be matched directly by **nearest retention time** (`matching='rt'` in
`Analysis.calc_plate_yield` / `Analysis.calc_plate_conv`), so no retention-index alkane standard is required;
the legacy two-machine retention-index workflow remains available via `matching='ri'` (the default). If the
result folder's `.acaml` metadata file is missing (e.g. an incomplete export), the FID injections are
enumerated directly from the `AIA/*_FID1A.cdf` files.

### Starting-material conversion and remaining starting material

In addition to product yields (`Analysis.calc_plate_yield`), pyGecko can quantify a **starting material**
relative to the internal standard:

- `Analysis.calc_plate_conv` reports **conversion** (`100 - remaining%`). By default it assumes the substrate
  was charged at the same loading as the internal standard (1 equiv); for a substrate charged in excess pass
  `equivalents` (e.g. `equivalents=1.5`) so its conversion is referenced to its actual starting amount instead
  of reading as a negative conversion. The result is floored at 0.
- `Analysis.calc_plate_rsm` reports the **remaining starting material** (the raw carbon-normalised area
  relative to the internal standard, in percent). It is reported as measured and never clamped, so an
  excess substrate can read above 100%.

See `examples/split_gc/` for a worked split-GC plate.

## Supported File Formats
pyGecko supports the following file formats:

| GC-MS         | GC-FID         |
|---------------|----------------|
| .mzML         | .xy            |
| .mzXML        | .CSV           |
| .D (Agilent)  | .cdf (ANDI/AIA)|
| .RAW (Thermo) ||
| .cdf (ANDI/AIA) ||

> [!NOTE]
> To achieve the best performance, we recommend using the .mzML file format for GC-MS data.

## Exporting Data

Processed injections and sequences can be written back out to open formats. MS data goes to mzML,
FID data to ANDI/AIA netCDF:

```python
from pygecko.parsers import (write_injection_to_mzml, write_sequence_to_mzml,
                             write_injection_to_cdf, write_sequence_to_cdf)

write_injection_to_mzml(ms_injection, 'FKB-FA-060-A1.mzML')
write_sequence_to_mzml(ms_sequence, 'exported/')       # one file per injection

write_injection_to_cdf(fid_injection, 'FBS-FA-033-A1.cdf')
write_sequence_to_cdf(fid_sequence, 'exported/')
```

mzML is written with [psims](https://github.com/mobiusklein/psims) and netCDF with netCDF4;
both ship with the default install.

> [!IMPORTANT]
> An export is a record of the injection **as pyGecko holds it**, not a copy of the original
> vendor file. pyGecko's readers round m/z to nominal integer mass and keep no polarity,
> instrument or acquisition metadata, so the MS1/centroid/positive terms in the written mzML are
> the writer's defaults rather than values from the source. Data written by pyGecko reads back
> through pyGecko's own readers unchanged; it is not a faithful round-trip of the raw file.

FID data is written as netCDF rather than mzML deliberately. The PSI-MS controlled vocabulary has
no term for a flame ionization detector, and none of its chromatogram types describes one, so an
mzML export of FID data would be schema-valid but semantically wrong. ANDI/AIA (ASTM E1947/E1948)
is the chromatography standard for a detector trace, and pyGecko already reads it.

## How to Cite

If you use pyGecko in your research, please cite the following publication:

**Calibration-free quantification and automated data analysis for high-throughput reaction screening**
Felix Katzenburg, et al.
*Digital Discovery*, 2025, **4**, 384-394.
DOI: [10.1039/D4DD00347K](https://doi.org/10.1039/D4DD00347K)
