Metadata-Version: 2.4
Name: GeneyxPgxParser
Version: 0.2.0
Summary: Convert PharmCAT / pbStarPhase / PyPGx / ONT / Thermo PharmacoScan PGx output into the Geneyx PGx JSON format.
Author: Bar Cohen
Author-email: Bar Cohen <bar@geneyx.com>
License: MIT License
        
        Copyright (c) 2026 Geneyx
        
        Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Project-URL: homepage, https://github.com/geneyx/GeneyxPgxParser
Project-URL: repository, https://github.com/geneyx/GeneyxPgxParser
Project-URL: documentation, https://geneyx.com/docs
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
Requires-Python: <3.15,>=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Dynamic: license-file

# GeneyxPgxParser

Converts pharmacogenomics caller output — PharmCAT, pbStarPhase, PyPGx, the
ONT PGx workflow, a Thermo Fisher PharmacoScan report, or a plain
Gene/Diplotype table — into the JSON format Geneyx Analysis consumes for PGx
reporting.

Install it next to your own PacBio, ONT or array pipeline, run it on the
output, and upload the result to Geneyx.

```bash
pip install GeneyxPgxParser
geneyx-pgx-parser HG002.pharmcat.report.json HG002.pbstarphase.json --output-json HG002.geneyx-pgx-results.json
```

## Output format

A JSON array, one entry per gene - **every gene the input files call**,
nothing dropped - with each value **as the vendor file writes it**:

```json
[
  {
    "Id": "CYP2D6",
    "Name": "CYP2D6",
    "StarAllele": "*1/*3",
    "Activity": "Intermediate Metabolizer",
    "ActivityScore": "1.0",
    "HaplotypeAlleles": null
  },
  {
    "Id": "IFNL3",
    "Name": "IFNL3",
    "StarAllele": "Reference/rs12979860",
    "Activity": "Unfavorable Response",
    "ActivityScore": null,
    "HaplotypeAlleles": [
      ["Reference"],
      ["rs12979860", "rs8099917", "rs12980275"]
    ]
  },
  {
    "Id": "HLA-DRB1",
    "Name": "HLA-DRB1",
    "StarAllele": "*04:01:01:10/*07:01:01:01",
    "Activity": null,
    "ActivityScore": null,
    "HaplotypeAlleles": null
  }
]
```

**The parser does not translate.** Since 0.2.0 mapping vendor terms to Geneyx
terms is done on the report side: `StarAllele` and `Activity` are the vendor's
own diplotype and phenotype, unchanged - PharmVar sub-alleles (`*2.001/*4.015`),
every HLA field, Thermo's tokens (`rs2231142G/rs2231142G`) and phenotype codes
(`IM`, `IM_or_PM`), PharmCAT's terms (`ivacaftor non-responsive in CF patients`),
no-call words (`./.`, `Unknown/Unknown`) and the caller's allele order included.

- **`Id`** / **`Name`** — the gene symbol. Both are required; the Geneyx report
  keys its result set on `Id` and filters on `Name`.
- **`StarAllele`** — the **diplotype** exactly as the caller wrote it, or
  `null` when the file names the gene without a call (an empty cell).
- **`Activity`** — the **phenotype** exactly as the caller wrote it (several,
  as PharmCAT lists for HLA-B, `; `-joined), or `null` when it gave none. The
  one phenotype the parser looks up itself is for a caller that runs no
  phenotyper (PharmCAT's matcher alone, the ONT CYP2D6 report): CPIC's
  diplotype table, the table PharmCAT's own phenotyper reads (see _ONT PGx
  output_).
- **`ActivityScore`** — the vendor's activity score as it writes it
  (PharmCAT's `activityScore`, or CPIC's table beside a looked-up phenotype),
  or `null`. The phenotype alone cannot always carry it: DPYD scores 0.0 and
  0.5 are both `Poor Metabolizer` and get different DPWG advice.
- **`HaplotypeAlleles`** — every allele the caller found on each of the two
  haplotypes, in the caller's order, or `null`. Only PyPGx lists them
  (`Haplotype1`/`Haplotype2`); its `Genotype` names each haplotype by one
  allele only, the first in its allele table, so in the IFNL3 example above
  rs8099917 and rs12980275 were called but appear nowhere else. The lists are
  in PyPGx's phase order, which need not be `StarAllele`'s order, and hold
  every allele whose definition the haplotype matches, so an allele that
  another contains appears beside it (`*1` with `*17`). A gene PyPGx calls
  from copy number alone (GSTT1, UGT2B17) has empty haplotype cells and `null`.

These names are matched case-sensitively on the Geneyx side. Do not rename them.
`ActivityScore` and `HaplotypeAlleles` are additions the Geneyx side does not
read yet; its JSON reader skips members it does not know, so they are safe to
carry. 0.2.0 removed `UnsupportedAlleles` from the array and `inReport` /
`reportNote` / `vendorDiplotype` from the provenance: they described the check
against the report's dataset, which belongs to the report now.

A second file, `<output>.provenance.json`, is written alongside the result. It
records the caller versions, the table versions read, and per-gene warnings.
It is not part of the upload contract — it exists so a report can be traced
back to what produced it. Each gene carries `source` (the caller its call came
from), `phenotypeSource` (CPIC / DPWG / PharmCAT / Thermo, or CPIC for a
looked-up phenotype) and `coverageCaveats`: what the caller could not see for
it, which the reported call does not rule out - PharmCAT's
`Missing_Positions` / `Uncallable_Haplotypes` (the matcher TSV) and
`uncalledHaplotypes` / variants without a call (the JSON), Thermo's
`Unknown Call` alternatives (GAN-9404). For NA19207 that is CYP2C9 `*6/*25/*85`,
G6PD (9 variants), NUDT15 `*6/*9/*18`, DPYD `*3/*7` and RYR1. The result is
unchanged by them. The sidecar's top level has `samples`, the sample each
input file names (PharmCAT's title, the matcher TSV's `Sample` column, the
ONT CYP2D6 report's `Sample` row, PyPGx's first column, Thermo's CEL file),
`warnings` for the run (a sample-ID mismatch between the files, or between a
file and the sample its name is for: NA19207's matcher TSV says `7NA19207`),
and `cpicMisses`, the matcher/ONT diplotypes CPIC's table had no phenotype for.
A PyPGx gene also carries `phaseExtended`: `true` when a called allele has
every defining variant absent from the 1KGP panel PyPGx phases with, so Beagle
dropped them and PyPGx placed the allele by its phase-extension heuristic
(`pypgx_panel_gaps.json`, a measured list per PyPGx release); `false` for any
other PyPGx call; `null` for a caller whose phase is not PyPGx's. The result
is unchanged by it.

## Supported inputs

| Input                       | Detected by               | Provides                                                                    |
| --------------------------- | ------------------------- | --------------------------------------------------------------------------- |
| `*.pharmcat.report.json`    | name or `pharmcatVersion` | diplotypes, phenotypes, tool + data versions (PharmCAT 2.x and 3.x)         |
| `*.pharmcat.phenotype.json` | name or `geneReports`     | diplotypes, phenotypes                                                      |
| `*.pbstarphase.json`        | name or `gene_details`    | diplotypes only; alone (PacBio WDL v4) the primary caller                   |
| `*.pbstarphase.tsv`         | `.tsv` header             | as the JSON (PacBio v4 summary)                                             |
| `<GENE>.tsv` (PyPGx)        | `.tsv` header             | diplotype + phenotype, one gene per file                                    |
| `*.pharmcat.tsv` (ONT)      | `.tsv` header             | diplotypes only (PharmCAT matcher; no CYP2D6)                               |
| `CYP2D6/report.tsv` (ONT)   | `Sample` first row        | CYP2D6 diplotype only                                                       |
| `*_phenotype.rpt` (Thermo)  | `.rpt` or `Known Call`    | diplotype + phenotype (PharmacoScan plate report; `--sample`)               |
| `*.csv` / `*.json` table    | fallback                  | Gene + Diplotype (+ optional Phenotype)                                     |

The minimal table is the upload format: a header with **Gene** and **Diplotype**
is enough. Column names are matched loosely (`gene`, `geneSymbol`, `symbol`;
`diplotype`, `genotype`, `starAllele`).

## Why two files are better than one

PharmCAT only reports genes that have a CPIC or DPWG guideline. In a PacBio
HG002 run, pbStarPhase called 35 genes and PharmCAT reported 22. Passing the
pbStarPhase summary alongside the PharmCAT output adds the other 13 — NAT2,
CYP1A2, CYP2A13, CYP2C8 and the HLA genes beyond HLA-A/B — with pbStarPhase's
diplotype and no phenotype (pbStarPhase writes none).

Where both files describe the same gene, PharmCAT wins: its call comes with
PharmCAT's phenotype and activity score.

### pbStarPhase on its own (PacBio WDL v4)

The HiFi human WGS WDL v4 dropped PharmCAT, so a v4 run has only the
pbStarPhase output. Then pbStarPhase is the **primary** caller: every gene it
calls is reported, as pbStarPhase wrote it, with `Activity: null` - the file
carries no phenotype, and since 0.2.0 the parser derives none (up to 0.1.8 it
ran PyPGx's phenotype step on the diplotype). Up to 0.1.5 the parser treated
pbStarPhase as PharmCAT's supplement even when it came alone, and a v4 sample
came out with 4 genes.

## ONT PGx output

The ONT PGx workflow writes its calls to two files, and both are needed:
`<sample>.pharmcat.tsv` holds every gene except CYP2D6, and
`CYP2D6/report.tsv` holds CYP2D6. Pass both, and they merge into one array:

```bash
geneyx-pgx-parser NA19207.pharmcat.tsv NA19207/CYP2D6/report.tsv --output-json NA19207.geneyx-pgx-results.json
```

The workflow runs only PharmCAT's allele matcher, not its phenotyper, so
neither file carries a phenotype. The Activity is looked up in CPIC's own
diplotype table - the table PharmCAT's phenotyper reads - packaged as
`Mappings/cpic_diplotypes.json`. The lookup tries both allele orders, writes
three or more copies the way CPIC does (`*2x3` -> `*2x≥3`), and falls back to
the haplotype columns where the matcher TSV abbreviates (CFTR); only the
lookup tries these spellings, `StarAllele` stays as the caller wrote it. CPIC's
phenotype is the `Activity` and its activity score the `ActivityScore`; which
CPIC key answered goes in the provenance sidecar.

DPYD is usually unphased: PharmCAT lists the variants it found rather than a
diplotype, and they are reported as-is, `;`-separated
(`c.85T>C (*9A);c.557A>G`). Two variants can only be one on each strand, so
the pair is looked up as a diplotype (the `StarAllele` keeps the `;`); three
or more need CPIC's lowest-two rule and are left without an Activity rather
than guessed.

Genes the table cannot answer keep `Activity: null`: CYP3A4 and CYP4F2 have
no CPIC diplotype table, G6PD needs the sample's sex, and IFNL3 / VKORC1 have
no CPIC diplotype table either.

`--input-dir` does not descend into `CYP2D6/`, so name the report explicitly.

### Keeping the CPIC table current

CPIC revises its tables in place - SLCO1B1 `*41` went from uncertain to no
function on 2025-09-23, so NA19207's `*37/*41` is `Decreased Function` here but
`Indeterminate` from PharmCAT 2.15.5, whose data predates the change.

The table is kept current in [pgx-metadata](#mapping-tables), where
`Test_CpicLiveTable.py` fetches CPIC and fails while the table disagrees
with it, listing every changed, added and removed diplotype and every
changed phenotype term. The refreshed table reaches this package as a new
mapping bundle.

### Running PharmCAT's phenotyper instead

The same Activity can come from running the rest of PharmCAT on the
workflow's `<sample>.match.json`, with the CYP2D6 call handed back to it as an
outside call. Use the same PharmCAT version as the workflow (wf-pgx v0.1.7
ships 2.15.5):

```bash
geneyx-pgx-parser NA19207/CYP2D6/report.tsv --pharmcat-outside-call NA19207.outside.tsv
pharmcat -phenotyper -pi NA19207.match.json -po NA19207.outside.tsv -reporter -reporterJson -o out -bf NA19207
geneyx-pgx-parser out/NA19207.report.json --output-json NA19207.geneyx-pgx-results.json
```

The `report.json` is the same file the PacBio path produces, and PharmCAT's
phenotypes are used as-is - no CPIC lookup. Unphased DPYD keeps its
`;`-separated variants as StarAllele and takes the phenotype PharmCAT gives
the pair.

## Thermo Fisher PharmacoScan

Axiom Analysis Suite's `apt-pgx-translation` writes one `*_phenotype.rpt` per
**plate**, so the file holds every sample on it while the Geneyx result is per
sample. Name the sample, by CEL file name or its stem:

```bash
geneyx-pgx-parser 2026-02-18-PharmacoScan24F_phenotype.rpt --sample HG03225 --output-json HG03225.geneyx-pgx-results.json
```

A file with one sample needs no `--sample`; a file with several and no choice
is refused with the list of samples it holds (GAN-8867).

What the parser does with the report:

- **`Known Call` is the `StarAllele` and `Phenotype Call` the `Activity`,
  both exactly as Thermo writes them** (`rs2231142G/rs2231142G` and `NF`,
  `Met/ValB` and `IM_or_PM`, `*1/*29` and `NM`). Thermo's allele vocabulary,
  its phenotype codes, a `,`-separated `Known Call` (Thermo could not phase the
  sample: `*1/*14,*4/*37`) and a copy-number gain (`(*1/*10)xN`) are the
  report's to read. Up to 0.1.8 the parser translated the tokens through
  `thermo_alleles.json` and replaced `Phenotype Call` with PyPGx's phenotype.
- **Rows with no `Known Call`** (`UNDH`, `CN_HybridLoss`) are written as
  no-calls (`StarAllele: null`), with the interpretation code and the
  `Unknown Call` candidates in the provenance.
- **`Unknown Call` beside a `Known Call` is a coverage caveat.** Control1's
  CYP2D6 `*1/*29` stays `*1/*29` NM; Thermo's alternatives with a haplotype
  the chip cannot resolve (`*2/UNK,*34/UNK,...`) go to the sidecar's
  `coverageCaveats` for the gene, and weigh nothing (GAN-9404).
- **An interpretation code that qualifies the call** (`CN_HybridGain` on
  `*2/*4`, `CN_Gain`, `CN_Error`) keeps the call as written and adds a warning
  saying what the code means; `CN_NoCall` (no copy number reported) adds a
  coverage caveat.
- **Every gene the chip calls is reported** (ALDH2, FMO3, TNF, UGT1A9 …). Only
  `CYP2CRS12777823` is left out: it is a carrier for the rs12777823 warfarin
  variant rather than a gene (`ignored_genes` in `thermo_alleles.json`, whose
  `gene_aliases` also give `MTRNR1` its gene symbol `MT-RNR1`). On the 24F
  panel that is 31 of the 32 reported genes.

A report records its own library as `thermoLibrary` in the provenance sidecar
(read from its `#%MetabolizerFile=` header); one made with a different library
than `thermo_alleles.json` names carries a warning on every gene.

## Merging rules

- **One entry per gene.** PharmCAT 2.x emits a gene report per phenotype
  source, so the same 22 genes appear under both CPIC and DPWG. One source is
  preferred and the other is consulted only as a fallback — see below.
- **One caller per gene.** Where several files call a gene, the higher-ranked
  caller wins (PharmCAT's report over its phenotype JSON over pbStarPhase /
  PyPGx / ONT), and its diplotype, phenotype and activity score are reported
  together: one caller's phenotype is never attached to another's diplotype.
- **No translation.** `StarAllele` and `Activity` are the winning caller's own
  values (see _Output format_); only PyPGx's trailing `;` on a cell is
  dropped. Several candidate diplotypes (pbStarPhase's list, PharmCAT's
  unphased DPYD variants) are reported `;`-separated, never the first alone.
- **A no-call is written, never dropped.** A gene the caller names but did not
  call is written with the caller's own word (`./.`, `Unknown/Unknown`,
  `Indeterminate`) as its `StarAllele`, or `null` for an empty cell, and no
  Activity. Where the preferred caller made no call and another caller in the
  run did, the other call is reported, with a warning saying so.
- **A placeholder is not a phenotype.** PharmCAT's `n/a` (a `no_call_tokens`
  value in `gene_rules.json`) is `Activity: null`. PyPGx's own values,
  `Indeterminate` and an empty cell included, are reported as PyPGx wrote them.

## Choosing a phenotype source

PharmCAT phenotypes come from two guideline bodies, **CPIC** and **DPWG**, and
they do not always agree. Pick one:

```bash
geneyx-pgx-parser HG002.pharmcat.report.json --phenotype-source DPWG
```

The default is **CPIC** (the first entry of `phenotype_source_preference` in
`gene_rules.json`). The choice only reorders the sources — it never narrows the
output. When the chosen source has no usable phenotype for a gene, the next
source in the default order is used instead, and the substitution is recorded:

```json
{
  "Id": "CYP2B6",
  "phenotypeSource": "CPIC",
  "warnings": ["no phenotype from DPWG; taken from CPIC"]
}
```

"No phenotype" covers two cases: the source reported none at all (HG002
CPIC: CYP3A4, CYP4F2, VKORC1) or it reported a placeholder such as `n/a`
(HG002 DPWG: CYP2B6). The other source's phenotype and activity score are
taken only for the same diplotype, as that source wrote them - a DPWG term
(`-1639 GG` for VKORC1) is reported as DPWG writes it.

The source selection affects `Activity` and `ActivityScore` only — `StarAllele`
is identical either way, since both sources describe the same call. The preference actually used is
written to the provenance file as `phenotypeSourcePreference`.

## PyPGx

The Geneyx PyPgx pipeline has produced this JSON format since it shipped, by
reading PyPGx's per-gene `data.tsv` and writing its `Genotype` and `Phenotype`
columns straight through. This parser does the same, as it now does for every
caller.

Each PyPGx file covers one gene, and the gene comes from the file name
(`CYP2C19.tsv`), because PyPGx's own table has no gene column. A table still
named `data.tsv` needs a `Gene` column, or parsing fails rather than reporting
a gene called "data". Point the parser at the whole results folder:

```bash
geneyx-pgx-parser --input-dir /path/to/pgx/results --output-json sample.geneyx-pgx-results.json --gzip
```

## Mapping tables

The clinical vocabulary lives in `GeneyxPgxParser/Mappings/` as versioned JSON.
The tables are **owned by pgx-metadata** (Azure DevOps), where reference
sources are snapshotted, mapping proposals are reviewed and approved, and a
signed, versioned **mapping bundle** is built. This repository pins one bundle
in `bundle.lock`; the packaged tables are exactly that bundle's, and
`Test_MappingBundle.py` fails if a table is edited here. To adopt a new bundle:

```bash
python scripts/sync_bundle.py --from <pgx-metadata bundle folder> --public-key bundle.pub
```

and open one pull request with `bundle.lock` and the tables it changed.
pgx-metadata runs the parser against its candidate tables by naming their
folder in `GENEYX_PGX_MAPPINGS_DIR`; unset, the packaged tables are used.

| File                    | Read for                                                                    |
| ----------------------- | --------------------------------------------------------------------------- |
| `gene_rules.json`       | phenotype source preference, gene scope, no-call tokens, CPIC-lookup callers |
| `cpic_diplotypes.json`  | the phenotype and activity score of a matcher / ONT call                    |
| `thermo_alleles.json`   | Thermo gene aliases, ignored pseudo-genes and the library it was built for  |
| `pypgx_panel_gaps.json` | `phaseExtended` in the provenance                                           |

The bundle still carries the term tables (`phenotype_terms.json`,
`genotype_terms.json`, `allele_terms.json`, `pypgx_phenotypes.json`,
`report_diplotypes.json`, and the allele and phenotype-code halves of
`thermo_alleles.json`); since 0.2.0 the parser does not read them. Mapping
vendor terms to Geneyx terms is the report side's job, and that is where
those tables belong.

## Development

```bash
pip install -e .
pytest GeneyxPgxParser/Test/Test_GeneyxPgxParser.py -v
```

`pip install -e .` compiles the modules in place, and Python loads a
`.pyd`/`.so` in preference to the `.py` beside it — so after an edit the tests
keep running the **copy built at install time** until you reinstall. The
symptom is a change that provably has no effect. Delete the build artefacts to
go back to running the sources:

```bash
find GeneyxPgxParser -name "*.pyd" -delete -o -name "*.so" -delete -o -name "*.c" -delete
```

They are gitignored, and CI builds the wheels from source regardless.

Test fixtures are real PacBio output for **HG002**, a public GIAB reference
sample, ONT output for **NA19207**, and Thermo's own PharmacoScan 24F demo
plate unmodified — 24 samples × 32 genes, two Thermo controls and 22
1000 Genomes / Coriell cell lines, as shipped with Axiom Analysis Suite. The
whole plate is kept rather than a cut-down copy because its awkward calls are
spread across samples: between them they cover every interpretation code the
format has. No patient data belongs in this repository.
