Metadata-Version: 2.4
Name: variant-hierarchy
Version: 0.1.0
Summary: Query precomputed cancer variant hierarchy relations with lightweight variant normalization.
Author: VariantHierarchy contributors
License: Apache-2.0
Keywords: cancer,variant,hierarchy,normalization,bioinformatics
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: vicc
Requires-Dist: requests>=2; extra == "vicc"
Provides-Extra: research
Requires-Dist: requests>=2; extra == "research"
Requires-Dist: pandas>=2; extra == "research"
Requires-Dist: PyYAML>=6; extra == "research"
Requires-Dist: pyparsing>=3; extra == "research"
Requires-Dist: hgvs; extra == "research"
Requires-Dist: cool-seq-tool; extra == "research"
Requires-Dist: biocommons.seqrepo; extra == "research"
Provides-Extra: dev
Requires-Dist: build; extra == "dev"
Requires-Dist: twine; extra == "dev"
Requires-Dist: pytest; extra == "dev"
Dynamic: license-file

# VariantHierarchy

VariantHierarchy normalizes classified cancer variant terms from COSMIC, OncoKB, CIViC, and CGI, then builds parent-child relations between normalized variants. The project is designed for evidence retrieval across variant granularities, for example allowing a rule attached to `EGFR exon 19 in-frame deletion` to match a more specific event such as `EGFR E746_A750del`.

## Python Package Runtime

The PyPI package is the lightweight runtime for querying precomputed variant hierarchy relations. It does not require SeqCAT, UTA, SeqRepo, pandas, or local database services for its default functionality.

```python
from variant_hierarchy import (
    ancestors,
    descendants,
    gene_hierarchy,
    hierarchy,
    normalize_variant,
    parents,
    variant_hierarchy,
)

normalize_variant("BRAF V600")
parents("BRAF V600D")
ancestors("EGFR E746_A750del")
descendants("BRAF V600")

# Gene hierarchy mode, matching the visualization's gene view.
gene_hierarchy("BRAF")

# Variant hierarchy mode, matching the visualization's node view.
variant_hierarchy("BRAF V600D")

# Auto mode: gene input returns a gene hierarchy; variant input returns a variant hierarchy.
hierarchy("BRAF")
hierarchy("BRAF V600D")
```

The runtime package includes only:

- `hierarchy_non_redundant.json`: precomputed non-redundant hierarchy edges;
- `gene_aliases.tsv`: local zero-deployment gene alias normalization;
- `aa_position_aliases.tsv`: amino-acid site labels derived from hierarchy nodes, replacing runtime SeqCAT calls.

Research and build workflows remain in the repository and may use SeqCAT, UTA, source database files, and heavier dependencies to regenerate package data. After regenerating research outputs, refresh package data with:

```bash
python scripts/export_package_data.py
```

## Core Model

The central data structure is `Variant` in `normalize/variant.py`. Each normalized variant has one display name and three semantic dimensions:

| Field | Meaning | Examples |
| --- | --- | --- |
| `location` | Where the variant is described. | `{"gene": "EGFR"}`, `{"gene": "EGFR", "exon": "19"}`, `{"gene": "BRAF", "amino_acid": "600"}` |
| `alteration` | What molecular alteration is described. | `mutation`, `missense`, `inframe_deletion`, `amplification`, `fusion`, `overexpression` |
| `function` | Functional or oncogenic interpretation. | `gain_of_function`, `loss_of_function`, `oncogenic mutation`, `inconclusive mutation` |

`variant_class` is the top-level category used to organize normalized variants:

| `variant_class` | Scope | Examples |
| --- | --- | --- |
| `HGVS Variant` | Concrete protein-level variants with a specific `p_hgvs_posedit`. | `BRAF V600E`, `EGFR E746_A750del`, `KIT K558_V559del` |
| `Sequence Variant` | Sequence-level categories broader than a concrete HGVS event. | `TP53 mutation`, `BRAF V600X`, `EGFR exon 19 in-frame deletion`, `BRCA2 truncating mutation`, `MET exon 14 skipping mutation` |
| `Copy Number Variant` | Copy-number gain/loss categories. | `ERBB2 amplification`, `CDKN2A copy number deletion`, `VHL exon 1 copy number deletion`, `PTCH1 loh` |
| `Structural Variant` | Gene rearrangement and fusion categories. | `ALK fusion`, `EML4::ALK fusion`, `BRAF rearrangement`, `ERBB4 translocation` |
| `Expression Variant` | Expression-state variants. | `ERBB3 absent`, `CASP8 overexpression`, `ATM underexpression` |
| `Epigenetic Variant` | DNA methylation-state variants. | `MLH1 methylation`, `NAPRT hypermethylation`, `CCND2 demethylation` |
| `Function Variant` | Function or oncogenicity classes without a required physical alteration. | `PTEN loss-of-function`, `EGFR gain-of-function`, `BRAF oncogenic mutation`, `IDH1 switch-of-function` |
| `Uncertain Variant` | Parsed as a retained but unresolved term. | Terms outside the current ontology/parser scope when explicitly retained. |

## Ontologies

The controlled vocabularies are defined in `util/ontology.py`.

`VARIANT_LOCATION_ONTOLOGY` defines coarse location containment, currently centered on gene, exon, and amino-acid locations.

`VARIANT_ALTERATION_ONTOLOGY` defines alteration normalization and parent mechanisms. Its parent links allow broad classes to contain narrower classes, for example:

- `mutation` contains `missense`, `truncating`, `splice`, `copy_number_change`, `rearrangement`, `expression`, and `methylation`.
- `truncating` contains `frameshift` and `nonsense`.
- `copy_number_change` contains `copy_number_gain` and `copy_number_loss`; these contain `amplification` and `copy number deletion`.
- `rearrangement` contains `fusion` and `translocation`.
- `methylation` contains `hypermethylation` and `demethylation`.

`VARIANT_FUNCTION_ONTOLOGY` defines functional parent-child logic, for example:

- `gain-of-function` contains `activating mutation`.
- `loss-of-function` contains `inactivating mutation`.
- `oncogenic mutation` contains `likely oncogenic mutation`.

## Normalization

The main parser is `normalize/variant_parser.py`. It converts source-specific terms into the `Variant` model by identifying the gene/reference, location granularity, alteration mechanism, and functional annotation.

The parser currently covers:

- protein-level substitutions, indels, duplications, frameshifts, nonsense variants, and site/wildcard terms;
- gene-, exon-, and amino-acid-region sequence categories;
- copy-number alterations including amplification, copy-number deletion, exon-level copy-number deletion, and loss of heterozygosity;
- fusion, rearrangement, and translocation terms;
- splice-site, splice-donor, splice-acceptor, splice-region, and exon-skipping terms;
- expression and epigenetic terms;
- functional and oncogenic class terms.

Unsupported terms are tracked separately when they require a future parser or ontology decision, such as rsIDs, c.HGVS-only descriptions, cytoband alterations, protein state/assay terms, invalid gene symbols, or ambiguous `Loss` terms.

## Hierarchy Construction

Hierarchy construction is implemented in `hierarchy/build_hierarchy.py`. For each candidate parent-child pair, the project compares three dimensions:

| Dimension | Implementation | Relation values |
| --- | --- | --- |
| Location | `hierarchy/match_position.py` | `equal`, `contains`, `unrelated` |
| Alteration | `hierarchy/match_type.py` | `equal`, `contains`, `unrelated` |
| Function | `hierarchy/match_function.py` | `equal`, `contains`, `unrelated` |

A parent contains a child when none of the three dimensions conflict and at least one dimension is broader. This supports relations such as:

- amino-acid site or wildcard variant contains a concrete protein variant: `BRAF V600X` contains `BRAF V600E`;
- exon or region-level sequence variant contains a concrete protein variant: `EGFR exon 19 in-frame deletion` contains `EGFR E746_A750del`;
- broader exon or region-level variant contains a narrower region-level variant: `EGFR exon 20 insertion` contains `EGFR exon 20 in-frame insertion`;
- gene-level mutation contains more specific sequence variants: `TP53 mutation` contains `TP53 R175H`, `TP53 R175X`, and `TP53 exon 5 mutation`;
- gene-level mutation contains compatible categorical variants through the alteration ontology: `ERBB2 mutation` contains `ERBB2 amplification`, and `ALK mutation` contains `ALK fusion`;
- functional class variant contains compatible mechanistic or functional categories: `PTEN loss-of-function` contains `PTEN truncating mutation`;
- fusion abstraction is an intended hierarchy target: `ALK fusion` should contain `EML4::ALK fusion` when location and function annotations are compatible; cases where parent-only function annotations block the edge are tracked by the hierarchy challenge set.

The hierarchy output is written to `data/hierarchy/hierarchy.json`.

## Project Data

- `data/variants/variants.json`: merged normalized variants from COSMIC, OncoKB, CIViC, and CGI.
- `data/hierarchy/hierarchy.json`: inferred hierarchy edges.
- `data/graph/node_with_info.csv` and `data/graph/edge.csv`: graph files used by Neo4j import and the D3 hierarchy visualization.
- `data/challenge_sets/normalization_challenge_set.tsv`: normalization challenge set.
- `data/challenge_sets/hierarchy_relation_challenge_set.tsv`: hierarchy relation challenge set.
- `data/challenge_sets/STRATA.md`: challenge-set column definitions and sampling strata.
- `data/challenge_sets/unsupported_terms_taxonomy.tsv`: unsupported-term taxonomy for error analysis.
- `evaluate/`: reproducible challenge-set evaluation and manuscript-ready SVG figures.

## Hierarchy Visualization

The D3.js visualization merged from the standalone `variant_hierarchy` project is
available at `visualization/`. It reads the graph files produced by
`hierarchy/hierarchy_class.py`.

Refresh the graph files:

```bash
python -c "from hierarchy.hierarchy_class import get_neo4j; get_neo4j(with_info=True)"
```

Serve the repository root and open the visualization:

```bash
python -m http.server 8000
```

```text
http://localhost:8000/visualization/
```

## Regenerating Challenge Sets

Run:

```bash
python scripts/build_challenge_sets.py
```

The script samples ten examples per challenge stratum and updates the TSV files and `data/challenge_sets/STRATA.md`.
By default it preserves an existing `normalization_challenge_set.tsv` so manually reviewed gold-standard rows are not overwritten. To regenerate that file explicitly, run:

```bash
python scripts/build_challenge_sets.py --rewrite-normalization
```

## Running Evaluation

Run:

```bash
python evaluate/evaluate_norm.py
python evaluate/evaluate_hier.py
```

The evaluation writes row-level results, per-stratum statistics, and a hierarchy confusion matrix to `evaluate/results/`.
