Metadata-Version: 2.4
Name: merge-cli
Version: 3.17.0
Summary: MERGE variant pathogenicity prediction CLI: 4 variant types, 14 model features, local/remote modes, MCP server
Author: MERGE Team
License-Expression: MIT
Project-URL: Homepage, https://merge.fanglab.cn
Keywords: bioinformatics,variant,pathogenicity,genomics,cli
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Environment :: Console
Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: THIRD_PARTY_NOTICES.md
Requires-Dist: click>=8.1
Requires-Dist: rich>=13.0
Requires-Dist: requests>=2.31
Requires-Dist: numpy>=1.26
Requires-Dist: pandas>=2.1
Requires-Dist: scikit-learn>=1.8
Requires-Dist: joblib>=1.3
Requires-Dist: matplotlib>=3.8
Requires-Dist: shap>=0.44
Requires-Dist: catboost>=1.2
Requires-Dist: xgboost>=3.0
Provides-Extra: mcp
Requires-Dist: mcp<2,>=1.2; extra == "mcp"
Provides-Extra: local
Requires-Dist: pysam>=0.22; extra == "local"
Provides-Extra: all
Requires-Dist: merge-cli[local,mcp]; extra == "all"
Dynamic: license-file

# merge-cli

MERGE variant pathogenicity prediction CLI — the professional-user counterpart to
[merge.fanglab.cn](https://merge.fanglab.cn), with the same models and the same
ensemble, runnable entirely on your own machine.

**New here? Read [TUTORIAL.md](TUTORIAL.md)** — install, both modes, per-model
setup, scripting and troubleshooting.

## Licence and terms

**The code is MIT** — commercial use, modification and redistribution all
permitted, no field-of-use restriction.

**The four bundled `MERGE_noAlphaGenome_*.pkl` ensembles are MIT too.** They are
fitted on a candidate pool with every `alphagenome_*` column removed, so no
AlphaGenome output went into training them.

Two things are narrower than that, and both come from upstream, not from us:

- **The AlphaGenome percentile table (`alphagenome_reference_quantiles.json`) is
  CC BY-NC 4.0** (non-commercial). It maps an AlphaGenome score to its percentile
  among the benchmark variants — no labels, nothing fitted, so no model is
  trained on AlphaGenome output anywhere in MERGE. It is used only for the
  optional MERGE + AlphaGenome average on splicing and non-coding variants —
  delete it and MERGE runs unchanged, reporting `merge_alphagenome_average: null`.
  See [LICENSE](LICENSE).
- **Four upstream components restrict commercial use**: AlphaGenome
  (non-commercial API), Nucleotide Transformer v1 and v2 (CC BY-NC-SA 4.0),
  dbNSFP (CC BY-NC-ND 4.0, paid commercial licence) and ANNOVAR (academic /
  non-profit only). Everything else MERGE calls — Evo 2, Carbon-3B, GENERator,
  HyenaDNA, GPN-MSA, Enformer, AlphaMissense, ESM-1b — is Apache-2.0, MIT,
  BSD-3 or CC BY 4.0.

Skipping a restricted model is supported: its feature is median-imputed instead
of erroring, so `--no-nt --no-nt-v2` plus an explicit `--ensemble-type` gives a
configuration with no non-commercial model dependency.

**Full details, per-component licences and the citations each one asks for:
[THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md).** MERGE redistributes none of
these models or databases — you obtain each yourself, under its own terms.

### AlphaGenome needs your own API key

In **both** remote and local mode. Variants already in the MERGE precomputed
cache return AlphaGenome values without a key; only a cache miss triggers a live
call, and that call is made under your key, never a MERGE-owned one.

```bash
merge alphagenome configure --api-key YOUR_KEY   # https://deepmind.google.com/science/alphagenome
```

> The AlphaGenome API is provided for non-commercial use only and is subject to
> the AlphaGenome Terms of Service. Outputs generated by AlphaGenome should not
> be used for the training of other machine learning models.
>
> Predictions are for theoretical modelling and research purposes only; they
> should not be used for clinical decision-making or relied upon for medical or
> other professional advice.

### Not for clinical use

MERGE is a research prototype — **not** a medical device, not clinically
validated, no regulatory approval. Its output must not drive diagnosis,
prognosis, treatment decisions or genetic counselling. Variant interpretation
remains the responsibility of qualified professionals following guidelines such
as ACMG/AMP. No warranty; see [LICENSE](LICENSE).

## Highlights

- **Four variant types**: coding, splice, noncoding and **non-SNV** (indel / MNV / delins).
- **13 models**: AlphaGenome, HyenaDNA, NT, AlphaMissense, ESM-1b, GPN-MSA,
  Evo2-7B, Evo2-7B-base, Enformer, GENERATOR, GENERATOR-v2, NT-v2, Carbon-3B.
  (Evo2-1B-base is also supported locally where the GPU allows it.)
- **Remote and local modes.** Local mode deploys each model into its own conda
  environment and serves it over HTTP — no Docker, no gateway required.
- Local precomputed VCF cache lookup for hg38/hg19 coding and splicing SNVs
  (disabled in this release; see "Precomputed VCF Cache").
- Bundled MERGE ensemble models (the same four deployment bundles the website runs).

## Quick Start

```bash
pip install merge-cli
merge --help
merge predict --chrom chr1 --pos 69428 --ref T --alt G --genome hg38
```

Optional extras: `merge-cli[mcp]` to ask for predictions in chat (see below),
`merge-cli[local]` for local mode (adds pysam, which has no Windows wheels —
which is why it is not in the base install), or `merge-cli[all]` for both.

## Ask in chat (MCP)

```bash
pip install "merge-cli[mcp]"
merge mcp install       # 1. register the tools  (Claude Desktop / Claude Code / Codex)
merge skill install     # 2. so the agent reaches for them on its own
merge mcp status
```

On Windows, `merge` is usually *"not recognized"* right after installing: pip
puts `merge.exe` in `...\Python\Scripts\` and warns that the directory is not on
PATH. Use `python -m merge_cli` instead of `merge` — the registration it writes
is correct either way, because MCP clients launch the server by absolute path.

Both steps are needed and neither replaces the other: `mcp install` is what
makes the tools exist, `skill install` is what makes the model notice them. A
registered server that the agent never thinks to use answers from memory
instead, silently.

Then restart your client and ask *"Is chr1:1040819 G>GC pathogenic?"*. The MCP
server runs in remote mode only — no GPU needed. Details in
[TUTORIAL.md §11](TUTORIAL.md).

## Variant types and ensemble models

The task is chosen automatically: a variant whose REF or ALT is not a single base
is scored as non-SNV; otherwise ANNOVAR decides coding / splice / noncoding.
Override with `--ensemble-type coding|splice|noncoding|nonsnv`.

| Task | Bundle | Algorithm | Features |
|---|---|---|---|
| coding | `MERGE_noAlphaGenome_coding.pkl` | LR_L2 | 6 |
| splice | `MERGE_noAlphaGenome_splicing.pkl` | CatBoost | 8 |
| noncoding | `MERGE_noAlphaGenome_noncoding.pkl` | HistGB | 12 |
| non-SNV | `MERGE_noAlphaGenome_nonsnv.pkl` | XGBoost | 12 |

Each bundle embeds its own preprocessing (median imputation → z-scoring →
classifier), so raw model scores are fed straight in and any model you did not
run is median-imputed. See `merge_cli/data/models/MODEL_CARD.md` for the exact
feature panels.

## The scores: MERGE, and the MERGE + AlphaGenome average

| | What it is | Which variants | Bands |
|---|---|---|---|
| **MERGE** | The ensemble probability. No AlphaGenome feature is in any panel. | all four classes | 0.8 / 0.6 / 0.4 / 0.2 |
| **MERGE + AlphaGenome average** | `0.5 x MERGE + 0.5 x percentile of one AlphaGenome column` | splicing and non-coding only | its own, per task |

Every prediction returns the MERGE ensemble score. Splicing and non-coding
variants return a second number as well: the MERGE + AlphaGenome average, a
fixed 1:1 mean of the ensemble score and the percentile of one AlphaGenome
column (`alphagenome_splicing` for splicing, `alphagenome_raw_score_max` for
non-coding) among the MERGE benchmark variants. The percentile comes from a
lookup table (`alphagenome_reference_quantiles.json`); nothing is fitted to
AlphaGenome output.

**Coding and non-SNV variants get the ensemble score only.** Their best single
AlphaGenome feature reaches AUROC 0.571 and 0.519 on held-out data, close to
chance. Those two classes return `merge_alphagenome_average: null` with status
`not_offered_for_this_variant_type`. On the benchmark (chromosome-held-out,
mean over three folds) the average reaches AUROC 0.9565 on splicing and 0.9793
on non-coding variants, against 0.9265 and 0.9472 for MERGE alone.

AlphaGenome is still called and reported in full for every variant class — all
seven summaries appear in the output. It does not feed the ensemble.

**The two scores are not on the same scale, and the average has its own
cut-points.** The per-task cut-points ship in `merge_final_thresholds.json`;
they are matched to the ensemble bands by the fraction of benchmark variants
in each band, without labels.

| Score | MERGE label |
|---|---|
| ≥ 0.8 | Pathogenic |
| ≥ 0.6 | Likely Pathogenic |
| ≥ 0.4 | Uncertain Significance |
| ≥ 0.2 | Likely Benign |
| < 0.2 | Benign |

The labels name score ranges, the same ones the web server shows. They are not
ACMG/AMP classifications and not calibrated probabilities.

When AlphaGenome has no value for a splicing or non-coding variant, the average
is reported as unavailable rather than silently falling back to the ensemble
score.

Full numbers: `merge_cli/data/models/MODEL_CARD.md`.

## Models

All models run by default; a model you skip has its feature median-imputed
wherever an ensemble panel uses it. AlphaGenome feeds only the MERGE + AlphaGenome
average, and NT-v2 is reported but not in any deployed panel. Skip any with
`--no-<model>`:

```bash
merge predict --chrom chr1 --pos 69428 --ref T --alt G --no-carbon --no-nt-v2
```

Available switches: `--no-alphagenome --no-hyenadna --no-nt --no-alphamissense
--no-esm1b --no-gpn-msa --no-evo2 --no-enformer --no-generator
--no-generator-v2 --no-nt-v2 --no-carbon`.

In local mode every model needs its service running; the CLI reports which
features were computed and which were imputed on every prediction.

### Evo2 variants

`--evo2-model` picks the weights for the primary Evo2 score (remote and local):

```bash
merge predict --chrom chr1 --pos 69428 --ref T --alt G --evo2-model evo2_7b_base
```

All three Evo2 variants are scored by default in local mode —
`evo2_7b_score`, `evo2_7b_base_score` and `evo2_1b_base_score` are separate
ensemble features. They share one GPU-resident service and are scored one at a
time, costing a few seconds each. Narrow the set to trade coverage for speed:

```bash
merge local predict --chrom chr1 --pos 69428 --ref T --alt G \
    --evo2-models evo2_7b        # primary variant only, faster
```

`evo2_1b_base` needs a GPU with FP8 (compute capability ≥ 8.9: RTX 4000/5000 Ada,
L40S, H100). Elsewhere — including in remote mode — its feature is left missing and
median-imputed by the ensemble, which is what the website does too.

## Local mode

```bash
merge doctor                                     # check GPU and services
merge local setup                                # configure paths
merge local env setup --model evo2               # deploy one model
merge local env setup --model carbon             # ... or another
merge local env setup --model all                # everything (large!)
merge local env start --model evo2
merge local env status
merge local predict --chrom chr17 --pos 43092919 --ref A --alt G
```

Setting a reference FASTA is strongly recommended — the local services use it to
cut the exact sequence windows the models were scored with during training:

```bash
merge local setup      # answer the "Reference genome FASTA path" prompt
# or
merge local predict ... --genome-ref /path/to/hg38.fa
```

Without it the services fall back to fetching sequence context from the MERGE
server, which requires network access.

### Service ports

| Model | Env | Port |
|---|---|---|
| AlphaGenome | `alphagenome` | 5000 |
| HyenaDNA | `hyenadna` | 5001 |
| NT | `nt` | 5002 |
| Evo2 (all variants) | `evo2` | 5003 |
| Enformer | `enformer` | 5004 |
| Carbon | `carbon` | 5005 |
| GENERATOR | `generator` | 5006 |
| GENERATOR-v2 | `generator_v2` | 5007 |
| NT-v2 | `nt_v2` | 5008 |

Each service exposes `GET /health` and `POST /predict` (`{chrom, pos, ref, alt,
genome_version}`); the DNA-LM services also expose `POST /batch_predict`.

Each model gets its own conda env, service and port by default, and there is no
gateway to stand up — so you can deploy only the models you actually need.

Environments may also be **shared**: any environment with the right dependencies
(torch + `transformers<5` + pyfaidx) can serve several of these models, which
saves tens of GB. Start the service with that environment's interpreter, e.g.
`~/miniconda3/envs/NT/bin/python ~/.merge-local-servers/generator_server.py`.
`merge local env status` shows which environment is really behind each port.

## Precomputed VCF Cache

```bash
merge precomputed configure --data-dir /path/to/precomputed
merge precomputed status
merge precomputed download --genome all --variant-type all
```

**Disabled in this release.** The VCF tables served for the cache
(`coding_merged.vcf.gz`, `splicing_merged.vcf.gz` and their hg19 counterparts)
predate the bundled models: they lack the newer model features and carry an
ensemble score from an earlier pipeline. Every variant is therefore scored with
the current models; `merge precomputed download` explains this and downloads
only with `--force`, and the cache is read only when `MERGE_ENABLE_PRECOMPUTED=1`
is set. Precomputed MERGE scores for the 107,922 coding and 1,920,006
splice-region candidate SNVs are distributed with the MERGE-benchmark archive.

## Scoring conventions

The local services reproduce the exact quantities the ensemble was trained on.
Changing any of them silently invalidates the MERGE score:

| Model | Score | Window | VRAM |
|---|---|---|---|
| Evo2-7B / -7B-base | mean_PLL(REF) − mean_PLL(ALT) | 8192 bp | ~20 GB |
| Evo2-1B-base | mean_PLL(REF) − mean_PLL(ALT) | 8192 bp | ~8 GB, needs FP8 |
| Carbon-3B | logP_sum(REF) − logP_sum(ALT) | 24576 bp | **~20 GB** |
| GENERATOR / -v2 | NLL(ALT) − NLL(REF) | 600 bp, max_length 128 | ~6 GB |
| NT-v2 | ‖emb(ALT) − emb(REF)‖₂ | 600 bp, max_length 128 | ~3 GB |
| NT | ‖emb(ALT) − emb(REF)‖₂ | 8192 bp | ~3 GB |
| HyenaDNA | log p(ALT) − log p(REF), next token | 1000 bp upstream | ~2 GB |

These windows are not tunable, on purpose. The MERGE bundles were fitted on
features computed exactly this way, so a shorter window does not merely cost
accuracy — it puts the feature on a different scale than the model expects, and
the resulting MERGE score is wrong without anything looking wrong. If a model
does not fit on your GPU, leave it off: the ensemble median-imputes what is
missing, which is the honest fallback.

Carbon in particular needs roughly 20 GB of VRAM at the 24576 bp training window.

The `transformers` version is pinned to 4.x in the generated environments:
NT-v2's remote code does not load under transformers 5.x. `alphagenome` is
pinned to 0.5.1 to match the MERGE server, so both modes return the same
AlphaGenome features.

AlphaGenome is the one model whose features are **not** reproducible over time:
it runs on Google's servers and is updated there, so today's values differ from
those in the training tables. Every other model reproduces bit-for-bit.

If local services are unavailable, the errors are reported under
`prediction.errors` and the CLI continues with the models it could reach.
