Metadata-Version: 2.4
Name: vhamster
Version: 1.4.1
Summary: VHAMSTeR ensemble inference CLI
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: click<9,>=8.3
Requires-Dist: loguru<1,>=0.7
Requires-Dist: numpy<3,>=2.4
Requires-Dist: polars<2,>=1.39
Requires-Dist: torch<3,>=2.11
Requires-Dist: tqdm<5,>=4.67
Requires-Dist: transformers<6,>=5.4
Requires-Dist: peft<1,>=0.18
Requires-Dist: biopython<2,>=1.87
Requires-Dist: pyrodigal-gv<0.4,>=0.3.2
Requires-Dist: scikit-learn<2,>=1.8
Requires-Dist: joblib<2,>=1.4
Requires-Dist: huggingface_hub<2,>=0.23
Requires-Dist: xgboost<3,>=2.1
Dynamic: license-file

# VHAMSTeR
![](vhamsterlogo.png)

[![PyPI version](https://img.shields.io/pypi/v/vhamster)](https://pypi.org/project/vhamster/)
[![PyPI downloads](https://img.shields.io/pypi/dm/vhamster)](https://pypi.org/project/vhamster/)
[![Conda downloads](https://img.shields.io/conda/dn/bioconda/vhamster)](https://anaconda.org/bioconda/vhamster)
[![install with bioconda](https://img.shields.io/badge/install%20with-bioconda-brightgreen.svg?style=flat)](https://bioconda.github.io/recipes/vhamster/README.html)

**V**irus **H**ost **A**ssignment **M**odel using **S**equence **T**ransformers and **R**eading-frames

VHAMSTeR uses a genomic language model to predict the host of a virus as one of: animal, plant, fungi, protist, or prokaryote.

It is designed for viral sequences up to 10 kbp. Longer sequences are split
into 10 kbp chunks (with a `_chunk<start>_<end>` suffix appended to the accession). 
Each chunk is scored independently, and the chunk predictions are mean-pooled to 
produce a genome-level consensus prediction.

## Dependencies

### Python packages (installed automatically)

- `click`
- `loguru`
- `torch`
- `transformers`
- `peft`
- `numpy`
- `polars`
- `biopython`
- `pyrodigal-gv`
- `tqdm`
- `scikit-learn`
- `xgboost`
- `joblib`

### External tools

- `mmseqs2` — used for geNomad marker search. If you use the conda environment (see below), it is included automatically. Otherwise install it via conda/mamba:

```bash
conda install -c bioconda mmseqs2
```

Alternatively, precompiled static binaries are available from the [MMseqs2 GitHub releases](https://github.com/soedinglab/MMseqs2/releases).

### geNomad database

A geNomad database is required at runtime. `vhamster-install-models` downloads it automatically from [Zenodo](https://zenodo.org/records/14886553). If you have an existing geNomad database you can point to it with `--genomad-db` instead.

## Installation

### Conda / Bioconda (Recommended)
The easiest way to install VHAMSTeR is through Bioconda, as it automatically installs both the Python package and the required `mmseqs2` external dependency.

```
conda install -c conda-forge -c bioconda vhamster
```

### PyPI (pip)
You can install the package directly from PyPI.

Note: mmseqs2 is not a Python package. If you install via pip, you must install mmseqs2 separately (see Dependencies above).

```
pip install vhamster
```

### Local / Development Install
If you need to modify the code, clone the repository and install it in editable mode:

```
git clone https://github.com/susiegriggo/VHAMSTeR.git
cd vhamster
pip install -e .
```

### Optional GPU support
CUDA is only needed if you want to run on GPU. CPU inference is supported and is often fast enough unless you are processing a large amount of data.

If you want GPU support, install a PyTorch build that matches your CUDA version. Visit https://pytorch.org/get-started/locally/ to get the right command for your setup. For example:

```
# CUDA 12.1
pip install torch --index-url [https://download.pytorch.org/whl/cu121](https://download.pytorch.org/whl/cu121)

# CPU only
pip install torch --index-url [https://download.pytorch.org/whl/cpu](https://download.pytorch.org/whl/cpu)
```

## Model installation

Model installation is a separate step after installing `vhamster` itself.

This downloads everything VHAMSTeR needs to run — the ensemble weights from [HuggingFace](https://huggingface.co/DOEJGI/vhamster-models), the geNomad marker database from [Zenodo](https://zenodo.org/records/14886553), and the NTv3 base transformer model from [InstaDeepAI](https://huggingface.co/InstaDeepAI/NTv3_650M_pre):

```bash
vhamster-install-models
```

During installation you will be asked to:

1. Agree to the [InstaDeepAI licence terms](https://huggingface.co/InstaDeepAI/NTv3_650M_pre) for the NTv3 base model (non-commercial use only)
2. Log in to HuggingFace — a browser tab will open automatically. If you do not yet have a free HuggingFace account, you can create one at [huggingface.co/join](https://huggingface.co/join) before running the command

Once installation is complete, vhamster runs fully offline — no HuggingFace account is needed at runtime.

By default, everything is installed into an environment-scoped location in the active Python
environment: `site-packages/vhamster_models_v1.4.0`.

If that default location is not writable, install to your own directory instead:

```bash
vhamster-install-models -o /path/to/my_vhamster_models
```

Reinstall everything if needed:

```bash
vhamster-install-models -f
```

Re-download a single component without touching the others:

```bash
# geNomad marker database only
vhamster-install-models --only genomad

# NTv3 base transformer model only
vhamster-install-models --only ntv3

# VHAMSTeR ensemble weights only
vhamster-install-models --only vhamster
```

`--only` can be repeated to combine components, e.g. `--only vhamster --only ntv3`.

The installer places files at:
- `<install_root>/fold_0/` … `<install_root>/fold_4/` — ensemble model weights
- `<install_root>/proportional_vector_scaling_scalar_nll_notclassbalanced_posthoc_fungi_nolength.json` — calibration parameters
- `<install_root>/genomad_db/` — geNomad marker database
- `<install_root>/base_models/` — NTv3 base transformer model

> **HPC / headless systems:** If no browser is available, either run `huggingface-cli login` in your terminal first, or pass your token directly:
> ```bash
> vhamster-install-models --hf-token $HF_TOKEN
> # or set the environment variable (safer — token won't appear in process listings):
> export HF_TOKEN=hf_xxxx
> vhamster-install-models
> ```

> **Cluster shared installations:** The base model only needs to be downloaded once per shared filesystem location. If a colleague has already installed into the same `<install_root>`, you do not need to run it again.

Once installed, vhamster will find the geNomad database automatically. If you have an existing geNomad database elsewhere, you can point to it with `--genomad-db`:

```bash
vhamster \
  --fasta input.fasta \
  --output results/ \
  --genomad-db /path/to/genomad_db
```

If your models are stored in a non-default location, pass the path with `--ensemble-dir`:

```bash
vhamster \
  --fasta input.fasta \
  --output results/ \
  --ensemble-dir /path/to/vhamster_models_v1.4.0
```

Runtime logs are written to `<output>/<prefix>.log` and also shown in the
terminal.

## Quick-start example

A test genome (accession NC_110914.1) is included in `test_data/`. After
installing `vhamster` and running `vhamster-install-models`, run from the
repository root:

```bash
vhamster \
  --fasta test_data/escherichia_phage.fasta \
  --output results/test_run \
  --prefix escherichia_phage
```

If your model files are stored in a non-default location:

```bash
vhamster \
  --fasta test_data/escherichia_phage.fasta \
  --output results/test_run \
  --prefix escherichia_phage \
  --ensemble-dir /path/to/vhamster_models_v1.4.0
```

This writes two files:

| File | Contents |
|------|----------|
| `results/test_run/escherichia_phage.chunks.tsv` | Per-chunk predictions |
| `results/test_run/escherichia_phage.genomes.tsv` | Genome-level consensus |

The `predicted_host` column in the genome file should read **Prokaryote** for
this *Escherichia* phage.

---

## Run

To see all available options:

```bash
vhamster --help            # common options
vhamster --help-advanced   # all options including advanced/expert ones
```

Minimal example (models at their default installed location):

```bash
vhamster \
  --fasta /path/to/input.fasta \
  --output /path/to/results_dir
```

Add a custom output prefix:

```bash
vhamster \
  --fasta /path/to/input.fasta \
  --output /path/to/results_dir \
  --prefix sampleA
```

Use a custom ensemble directory:

```bash
vhamster \
  --fasta /path/to/input.fasta \
  --output /path/to/results_dir \
  --ensemble-dir /path/to/vhamster_models_v1.4.0
```

Run using a single fold (for example, only `fold_3`):

```bash
vhamster \
  --fasta /path/to/input.fasta \
  --output /path/to/results_dir \
  --fold-index 3
```

Use a non-default calibration parameters file:

```bash
vhamster \
  --fasta /path/to/input.fasta \
  --output /path/to/results_dir \
  --calibration-params /path/to/calibration.json
```

## Two-stage pipeline (HPC)

For large datasets, the CPU-intensive feature extraction (PyRodigal + MMseqs2) and the GPU-intensive GLM inference can be run as two separate jobs. This lets you pipeline batches: one batch's features are computed on a CPU node while the previous batch's GLM inference runs on a GPU node.

**Stage 1 — feature extraction (CPU node, no GPU needed):**

```bash
vhamster-features \
  --fasta batch_001.fasta \
  --output features/batch_001 \
  --prefix batch_001
```

This writes:
- `features/batch_001/batch_001.arch_features.tsv` — per-chunk architectural features
- `features/batch_001/batch_001.genomad_hits.json` — geNomad marker hits
- `features/batch_001/batch_001.viral_marker_genes.tsv` — per-gene annotations

**Stage 2 — GLM inference (GPU node):**

```bash
vhamster \
  --fasta batch_001.fasta \
  --output results/batch_001 \
  --prefix batch_001 \
  --precomputed-features features/batch_001
```

The `--chunk-size` and `--overlap` values must match between the two stages (defaults are the same, so no flags needed if you use defaults for both).


## Outputs

For prefix `sampleA`, these files are always produced:

| File | Contents |
|------|----------|
| `sampleA.chunks.tsv` | Per-chunk host predictions with calibrated class probabilities |
| `sampleA.genomes.tsv` | Genome-level consensus (mean-pooled over chunks) |
| `sampleA.folds.tsv` | Per-fold calibrated predictions and GLM gate weights for all 5 ensemble members |
| `sampleA.viral_marker_genes.tsv` | Genes with a geNomad viral marker hit (subset of all predicted genes) |
| `sampleA.log` | Full run log |

Optional additional files (pass one or both flags):

| Flag | File | Contents |
|------|------|----------|
| `--model-scores` | `sampleA.model_scores.tsv` | Per-fold XGBoost, GLM and uncalibrated ensemble probabilities — use this to inspect individual model component scores |
| `--save-features` | `sampleA.features.tsv` | Input features fed to the model per chunk — architectural stats and geNomad marker features |

### Chunk Naming Convention
Sequences longer than the specified chunk size (default 10 kbp) are split into smaller fragments. The `accession` column for these fragments will include a `_chunk<start>_<end>` suffix (e.g., `NC_007026.1_chunk0_10000`). You can use this suffix or the `sampleA.genomes.tsv` file to join chunk-level predictions back to your original input sequences.

### File Schemas

**Chunk file columns include:**
- `accession`, `predicted_host`, `confidence`
- calibrated class probability columns
- `prokaryote_score`, `eukaryote_score`

**Genome file columns include:**
- `genome`, `predicted_host`, `confidence`
- calibrated class probability columns
- `prokaryote_score`, `eukaryote_score`

**Folds file columns include:**
- `accession`, `fold`, `predicted_host`, `confidence`, `glm_gate_weight`
- calibrated class probability columns

**Model scores file (`--model-scores`) columns include:**
- `accession`, `fold`, `n_genes`, `predicted_host`, `confidence`, `glm_gate_weight`
- calibrated class probability columns
- `xgb_<class>` (pure, uncalibrated XGBoost probabilities)
- `glm_<class>` (pure, uncalibrated GLM probabilities)
- `uncalibrated_ensemble_<class>` (the exact mathematical output of the dynamic gate before temperature scaling)

**Features file (`--save-features`) columns include:**
- `accession`
- All architectural features (gene density, fragment size, etc.)
- All geNomad marker features

## Performance & Batching Tips

**1. Batch your inputs into a single FASTA file**
The geNomad marker search relies on MMseqs2, which has a high fixed startup cost (often 1–2 minutes) to load the marker database into memory. This cost is incurred every time `vhamster` runs. 

To avoid paying this startup penalty multiple times, **do not run vhamster in a bash loop over individual files**. Instead, concatenate your sequences into a single multi-FASTA file and run `vhamster` once:

```
# Inefficient (Loads DB 3 times)
vhamster --fasta seq1.fasta --output out/ --prefix seq1
vhamster --fasta seq2.fasta --output out/ --prefix seq2
vhamster --fasta seq3.fasta --output out/ --prefix seq3

# Highly Efficient (Loads DB once)
cat seq1.fasta seq2.fasta seq3.fasta > all_seqs.fasta
vhamster --fasta all_seqs.fasta --output out/ --prefix all_seqs
```

## Citation 
**Preprint coming soon!** 

In the meantime you can cite 
* Grigson S. R., Roux, S. (2026). VHAMSTeR: Virus Host Assignment Model using Sequence Transformers and Reading-frames (v1.4.0).  [https://github.com/susiegriggo/VHAMSTeR](https://github.com/susiegriggo/VHAMSTeR)

If you use VHAMSTeR in your research, please also cite the foundational tools that power our feature extraction pipeline:

* **geNomad (Viral marker database and specificity concepts)** \
  Camargo, A. P., Roux, S., Schulz, F., Babinski, M., Xu, Y., Hu, B., Chain, P. S. G., Nayfach, S., & Kyrpides, N. C. (2023). Identification of mobile genetic elements with geNomad. *Nature Biotechnology*, 42, 277-280. [https://doi.org/10.1038/s41587-023-01953-y](https://doi.org/10.1038/s41587-023-01953-y)

* **Pyrodigal-gv / Prodigal (Viral gene prediction)** \
  Larralde, M. (2022). Pyrodigal: Python bindings and interface to Prodigal. *Journal of Open Source Software*, 7(72), 4296. [https://doi.org/10.21105/joss.04296](https://doi.org/10.21105/joss.04296)
  Hyatt, D., Chen, G. L., LoCascio, P. F., Land, M. L., Larimer, F. W., & Hauser, L. J. (2010). Prodigal: prokaryotic gene recognition and translation initiation site identification. *BMC Bioinformatics*, 11(1), 1-11. [https://doi.org/10.1186/1471-2105-11-119](https://doi.org/10.1186/1471-2105-11-119)

* **MMseqs2 (Fast protein sequence search)** \
  Steinegger, M., & Söding, J. (2017). MMseqs2 enables sensitive protein sequence searching for the analysis of massive data sets. *Nature Biotechnology*, 35(11), 1026-1028. [https://doi.org/10.1038/nbt.3988](https://doi.org/10.1038/nbt.3988)

## License Agreement
Lawrence Berkeley National Laboratory 
NON-COMMERCIAL USE ONLY LICENSE
 
V-HAMSTeR Copyright (c) 2026, The Regents of the University of California, through Lawrence Berkeley National Laboratory (“Berkeley Lab”) subject to receipt of any required approvals from the U.S. Dept. of Energy.  All rights reserved.
 
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
 
(1) Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
 
(2) Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
 
(3) Neither the name of the University of California, Berkeley Lab, U.S. Dept. of Energy nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission of Berkeley Lab.
 
(4) Use of the software, in source or binary form is for NON-COMMERCIAL USE purposes ONLY. The software is not available for commercial use. If you have any questions regarding this software, please contact Berkeley Lab at IPO@lbl.gov.

(5) User agrees to indemnify, defend, and hold harmless Berkeley Lab, the U.S. Government, the software developers, the software sponsors, and their agents, officers, and employees, against any and all claims, suits, losses, damage, costs, fees, and expenses arising out of or in connection with this Agreement.  User agrees to pay all costs incurred by Berkeley Lab in enforcing this provision, including reasonable attorney fees.

(6) In the event User creates any bug fixes, patches, upgrades, updates, modifications, derivative works or enhancements to the source code or binary code of the software ("Enhancements") User hereby grants Berkeley Lab and the U.S. Government a paid-up, non-exclusive, irrevocable, worldwide license in the Enhancements to reproduce, prepare derivative works, distribute copies to the public, perform publicly and display publicly, and to permit others to do so.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

## Copyright Notice 
V-HAMSTeR Copyright (c) 2026, The Regents of the University of California, through Lawrence Berkeley National Laboratory (“Berkeley Lab”) subject to receipt of any required approvals from the U.S. Dept. of Energy.  All rights reserved.
If you have questions about your rights to use or distribute this software, please contact Berkeley Lab's Intellectual Property Office at IPO@lbl.gov.
NOTICE: This Software was developed under Contract No. DE-AC02-05CH11231 with the Department of Energy (“DOE”). During the period of commercialization or such other time period specified by DOE, the U.S. Government is granted for itself and others acting on its behalf a nonexclusive, paid-up, irrevocable, worldwide license in the Software to reproduce, prepare derivative works, and perform publicly and display publicly, by or on behalf of the U.S. Government. Subsequent to that period, the U.S. Government is granted for itself and others acting on its behalf a nonexclusive, paid-up, irrevocable, worldwide license in the Software to reproduce, prepare derivative works, distribute copies to the public, perform publicly and display publicly, and to permit others to do so. The specific term of the license can be identified by inquiry made to Lawrence Berkeley National Laboratory or DOE. NEITHER THE UNITED STATES NOR THE UNITED STATES DEPARTMENT OF ENERGY, NOR ANY OF THEIR EMPLOYEES, MAKES ANY WARRANTY, EXPRESS OR IMPLIED, OR ASSUMES ANY LEGAL LIABILITY OR RESPONSIBILITY FOR THE ACCURACY, COMPLETENESS, OR USEFULNESS OF ANY DATA, APPARATUS, PRODUCT, OR PROCESS DISCLOSED, OR REPRESENTS THAT ITS USE WOULD NOT INFRINGE PRIVATELY OWNED RIGHTS.


