Metadata-Version: 2.4
Name: minerva-dna
Version: 0.2.2
Summary: Minerva: genomic language models for coevolutionary mining
Author: Garyk Brixi
License-Expression: Apache-2.0
Project-URL: Homepage, https://github.com/garykbrixi/minerva
Project-URL: Source, https://github.com/garykbrixi/minerva
Project-URL: Model, https://huggingface.co/gbrixi/minerva-mlm
Project-URL: Demo, https://huggingface.co/spaces/gbrixi/minerva
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: torch<3,>=2.1.0
Requires-Dist: transformers<6,>=4.41.0
Requires-Dist: numpy<3,>=1.23
Requires-Dist: biopython<2,>=1.80
Requires-Dist: pyrodigal<4,>=3.0
Requires-Dist: huggingface_hub<2,>=0.23.0
Requires-Dist: safetensors<1,>=0.4.0
Requires-Dist: matplotlib<4,>=3.7
Requires-Dist: ViennaRNA<3,>=2.6
Requires-Dist: datasets<5,>=2.16
Requires-Dist: accelerate<2,>=0.26
Requires-Dist: peft<1,>=0.10.0
Provides-Extra: flash
Requires-Dist: flash-attn>=2.5.8; extra == "flash"
Provides-Extra: viz
Requires-Dist: bokeh<4,>=3; extra == "viz"
Provides-Extra: dev
Requires-Dist: pytest<10,>=7; extra == "dev"
Requires-Dist: nbclient<1,>=0.8; extra == "dev"
Requires-Dist: ipykernel<8,>=6; extra == "dev"
Requires-Dist: ipywidgets<9,>=8; extra == "dev"
Dynamic: license-file


<picture>
  <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/garykbrixi/minerva/main/assets/minerva_banner_dark.png">
  <source media="(prefers-color-scheme: light)" srcset="https://raw.githubusercontent.com/garykbrixi/minerva/main/assets/minerva_banner_light.png">
  <img alt="Minerva: coevolutionary discovery using genome language models" src="https://raw.githubusercontent.com/garykbrixi/minerva/main/assets/minerva_banner_light.png">
</picture>

[Preprint](https://doi.org/10.64898/2026.09.22.753630) · [Model](https://huggingface.co/gbrixi/minerva-mlm) · [Demo](https://huggingface.co/spaces/gbrixi/minerva) · [PyPI](https://pypi.org/project/minerva-dna/)

Minerva predicts coevolution using genome language models. Powered by Minerva-MLM, it delivers database-scale, alignment-free, interaction-specific predictions across prokaryotic genomes. Through adaptation on homologous loci, Minerva can discover additional interactions.

Try it in the browser, no install needed: **[Minerva on Hugging Face Spaces](https://huggingface.co/spaces/gbrixi/minerva)**.

[Install](#install) · [Checkpoints](#pretrained-checkpoints) · [Quick start](#quick-start) · [Preparing inputs](#preparing-inputs) · [Interaction heads](#interaction-heads) · [Jacobian fingerprinting](#jacobian-fingerprinting) · [RNA structure](#rna-secondary-structure) · [Eukaryotic RNA](#eukaryotic-rna) · [Finetuning](#finetuning) · [Examples](https://github.com/garykbrixi/minerva/tree/main/examples/) · [Citation](#citation)

## Install

To install Minerva, use:

```bash
pip install minerva-dna
```

Minerva uses `flash-attn` automatically when it is installed. Otherwise it falls
back to PyTorch SDPA. Please install flash attention first for faster inference.

## Pretrained Checkpoints

Minerva-MLM is a 650M parameter transformer trained for over 1.3 trillion tokens (~3.4 Terabases). Minerva-MLM is initialized from [gLM2 650M](https://github.com/TattaBio/gLM2) and adopts the mixed-modality tokenization, and was trained at 4096 and 8192 context lengths.

Checkpoints are hosted on Hugging Face:

| Model       | Context | Hugging Face repo                                   |
| ----------- | ------- | --------------------------------------------------- |
| Minerva-MLM   | 4096    | [`gbrixi/minerva-mlm`](https://huggingface.co/gbrixi/minerva-mlm)         |
| Minerva-MLM-8k   | 8192    | [`gbrixi/minerva-mlm-8k`](https://huggingface.co/gbrixi/minerva-mlm-8k)   |

Minerva-MLM checkpoints include three interaction heads and Jacobian fingerprint types:

- **base_pairing** — RNA base-pairing contacts
- **protein** — protein contact prediction
- **repeat** — repeat element detection

## Quick start

```python
from transformers import AutoTokenizer
from minerva import MinervaForMaskedLM
import torch

model = MinervaForMaskedLM.from_pretrained(
    "gbrixi/minerva-mlm", torch_dtype=torch.bfloat16,
).cuda().eval()
tokenizer = AutoTokenizer.from_pretrained("gbrixi/minerva-mlm")

tokens = tokenizer(
    "<+>cgcggggtggagcagcctggtagctcgtcgggctcataacccgaagatcgtcggttcaaatccggcccccgcaacca",
    return_tensors="pt",
).to(model.device)

with torch.no_grad():
    outputs = model(**tokens, output_interactions=True)

base_pairing = outputs.interactions["base_pairing"]  # [batch, L, L]
protein = outputs.interactions["protein"]            # [batch, L, L]
repeat = outputs.interactions["repeat"]              # [batch, L, L]
```

> Importing the class directly needs no `trust_remote_code`. Without the package
> installed, use `AutoModelForMaskedLM.from_pretrained(repo, trust_remote_code=True)`.
>
> On Mac use `.to("mps")` instead of `.cuda()`. Keeping the model in float32 is
> compatible with more systems.

## Preparing inputs

Minerva reads a **mixed protein + DNA** sequence: coding regions are upper-case
amino acids, intergenic regions are lower-case nucleotides, and `<+>` / `<->`
markers denote strand.

```
<+>MALTKVEKRNRIKRRVRGK<+>aatttaaggaa<->MLGIDNIERVKPGGLELVDRLV
   └── CDS (protein) ──┘└ intergenic ┘└──── CDS on - strand ────┘
```

There are **three ways** to produce this format depending on what you start
with:

| You have | Use | What happens |
| --- | --- | --- |
| **Annotated GenBank** (CDS features) | `minerva.data.extract_and_tokenize_gb(path)` | CDS translated, intergenic kept as DNA, strand markers inserted. One sequence per LOCUS. |
| **Unannotated sequence** (FASTA / raw DNA) | `minerva.gene_calling.build_minerva_input(seq)` | Genes called with **Pyrodigal**, then packaged. |
| **Raw genome + external CDS calls** | `minerva.sequence_utils.build_prodigal_mixed_sequence(seq, cds)` | Your own `[{start, end, strand}]` calls packaged, with genome↔token maps. |

### From unannotated sequence (gene calling)

If you only have a FASTA file or a raw nucleotide string, we use [Pyrodigal](https://github.com/althonos/pyrodigal) to automatically call genes:

```python
from minerva.gene_calling import build_minerva_input, fasta_to_minerva_inputs

# From a single nucleotide string
out = build_minerva_input(sequence)          # dict: token_string + coord maps
token_string = out["token_string"]

# From a FASTA file (one result per record)
inputs = fasta_to_minerva_inputs("contigs.fasta")
```

Pyrodigal needs ≥ 20 kb to estimate gene-scoring statistics from a sequence;
shorter contigs use its pre-trained profiles, which `meta=True` forces
for metagenomic assemblies. `token_to_genome` / `genome_to_token` map character
index in `token_string` to genome position; for token index, use
`minerva.sequence_utils.build_token_to_genome_map`.

### Context length & capping

Minerva's context is 4096 (`gbrixi/minerva-mlm`) or 8192 tokens
(`gbrixi/minerva-mlm-8k`). One token is one amino acid, one nucleotide, or one
strand marker, so a typical (~88 % coding) bacterial genome packs to ~10 kb per
4096 tokens (~20 kb for the 8k model).

Pass `max_tokens` to the builders to cap a sequence.

```python
out = build_minerva_input(sequence, max_tokens=4096)   # <= 4096 tokens
```

To cover a whole genome, tile it instead with
`minerva.sequence_utils.chunk_sequence_with_stride`.

### Translation tables

CDS translate with NCBI table 11 by default, but a GenBank feature's own
`/transl_table` takes precedence. Override with `translation_table=` on the
builders or `--translation_table` on `scripts/finetune.py`.

See [`examples/`](https://github.com/garykbrixi/minerva/tree/main/examples/) for runnable, end-to-end walkthroughs.

## Using the model

These continue from the [quick start](#quick-start), with `model`, `tokenizer`, `tokens` and
`outputs` already defined.

### Interaction heads

To plot the interaction-head outputs:

```python
from minerva.visualization import plot_contacts

token_list = tokenizer.convert_ids_to_tokens(tokens["input_ids"][0].tolist())
plot_contacts(outputs, tokens=token_list, track=True)   # track=True adds a CDS / intergenic strip
```

`plot_contacts` takes the model outputs, a dict of head maps, or a Jacobian
`FingerprintResult` (below), and renders them all in the same palette. For a
zoomable view with hover values, `plot_contacts_interactive` takes the same
arguments and returns a Bokeh layout (`pip install "minerva-dna[viz]"`).

Set `interaction_layers=6` to use the six-layer interaction heads:

```python
outputs = model(**tokens, output_interactions=True, interaction_layers=6)
```

Raw attention tensors follow the Hugging Face convention:

```python
outputs = model(
    **tokens,
    output_interactions=True,
    output_attentions=True,
    attention_layers=[31, 32],
)
```

### Jacobian fingerprinting

Use `get_fingerprints` when you want named interaction-pattern channels from a
sequence. It computes the required full categorical Jacobian internally and returns
a `FingerprintResult`; the large raw Jacobian is not kept unless requested.

```python
sequence = "<+>cgcggggtggagcagcctggtagctcgtcgggctcataacccgaagatcgtcggttcaaatccggcccccgcaacca"
window = (0, min(len(sequence), 96))

fp = model.get_fingerprints(
    sequence,
    tokenizer,
    position_range=window,
    max_batch_size=64,
)

fp.channel_names                     # ["basepairing", "repeat", "protein", "other"]
basepairing = fp["basepairing"]      # [L, L]
protein = fp["protein"]              # [L, L]
repeat = fp["repeat"]                # [L, L]
```

To plot the result:

```python
from minerva.visualization import plot_contacts

plot_contacts(fp, tokens=fp.tokens, title="Minerva multimodal fingerprint")
```

### RNA secondary structure

The `base_pairing` head returns a dense contact map which can be converted to an RNA structure using `minerva.rna_structure`:

```python
from minerva.rna_structure import call_structure, call_structures

token_list = tokenizer.convert_ids_to_tokens(tokens["input_ids"][0].tolist())
s = call_structure(outputs.interactions["base_pairing"], tokens=token_list)

s.dot_bracket        # '(((((((..((((........)))).(((((.......)))))...'
s.to_vienna("t.fa")  # read by RNAfold, forna, VARNA, R2R
s.to_ct("t.ct")      # connect table, keeps pseudoknots
s.plot()             # matplotlib Figure
```

Each intergenic region of a mixed locus is a separate molecule, so
`call_structures` returns one structure per region:

```python
structures = call_structures(outputs.interactions["base_pairing"], token_list)
```

An interactive viewer is in
[`examples/notebooks/rna_structure.ipynb`](https://github.com/garykbrixi/minerva/blob/main/examples/notebooks/rna_structure.ipynb).

## Eukaryotic RNA

For researchers studying eukaryotic RNAs, Minerva provides a RiNALMo-based checkpoint with base-pairing and repeat interaction heads. See [eukaryotic RNA support](https://github.com/garykbrixi/minerva/tree/main/examples/eukaryotic_rna/) for usage and finetuning.

## Finetuning

`scripts/finetune.py` wraps HF `Trainer` + `accelerate` with Minerva's MLM loss.
It supports full finetuning and LoRA, and ingests GenBank files directly.

```bash
# LoRA finetune
accelerate launch --num_processes=8 scripts/finetune.py \
    --output_dir ./output \
    --genbank_file genome.gb \
    --tokenizer_name gbrixi/minerva-mlm \
    --model_name_or_path gbrixi/minerva-mlm \
    --use_lora --lora_r 1 --lora_alpha 2 \
    --learning_rate 1e-4 --bf16

# Full finetune on a GenBank file across 8 GPUs
accelerate launch --num_processes=8 scripts/finetune.py \
    --output_dir ./output \
    --genbank_file genome.gb \
    --tokenizer_name gbrixi/minerva-mlm \
    --model_name_or_path gbrixi/minerva-mlm \
    --per_device_train_batch_size 4 \
    --bf16
```

Minerva-MLM LoRA checkpoints loaded with PEFT:

```python
from peft import PeftModel
from minerva import MinervaForMaskedLM

base = MinervaForMaskedLM.from_pretrained("gbrixi/minerva-mlm")
model = PeftModel.from_pretrained(base, "path/to/lora_ckpt")
```

A LOCUS longer than `--max_seq_length` is split into non-overlapping blocks,
each its own training example, see
`minerva.finetuning.load_genbank_dataset`.

## Repo layout

```
minerva/
  modeling_minerva.py   # MinervaConfig / MinervaForMaskedLM (custom transformer + heads)
  modeling_rinalmo.py   # RiNALMoMinervaForMaskedLM (RNA backbone + heads)
  tokenization_rinalmo.py # RiNALMo nucleotide tokenizer
  backbones.py          # Backbone-specific training configuration
  interaction_heads.py # Shared interaction heads
  data.py               # GenBank parsing + tokenization
  gene_calling.py       # Pyrodigal gene calling: FASTA/raw DNA -> mixed tokens
  sequence_utils.py     # reverse-complement + external-CDS -> mixed tokens
  masking.py            # DataCollatorForMinervaMLM
  losses.py             # grouped_mlm_loss
  example_data/         # sample GenBank loci (minerva.data.example_path)
scripts/
  finetune.py                  # HF Trainer / accelerate wrapper
examples/                       # end-to-end tutorials & notebooks
  call_genes_from_fasta.py      # FASTA/raw DNA -> Minerva input walkthrough
  notebooks/                    # interactive Colab-ready notebooks
tests/                          # package unit + smoke tests
```

## Citation

If you use Minerva, please cite [Li & Brixi et al., *bioRxiv* 2026](https://doi.org/10.64898/2026.09.22.753630):

```bibtex
@article{li2026minerva,
  title     = {Coevolutionary mining of prokaryotic non-coding elements with a genome language model},
  author    = {Li, David B. and Brixi, Garyk and Kim, Alexandra S. and Fiamenghi, Mateus B. and Driscoll, Claudia L. and Evans, Simone A. and Gao, Alex and Ivanova, Natalia N. and Kyrpides, Nikos C. and Deisseroth, Karl and Wilkinson, Max E. and Fischbach, Michael A. and Hie, Brian L.},
  journal   = {bioRxiv},
  year      = {2026},
  doi       = {10.64898/2026.09.22.753630},
  url       = {https://www.biorxiv.org/content/10.64898/2026.09.22.753630},
  publisher = {Cold Spring Harbor Laboratory}
}
```

If you use the Jacobian fingerprints, please also cite the original categorical Jacobian, [Zhang & Wayment-Steele et al., *PNAS* 2024](https://doi.org/10.1073/pnas.2406285121):

```bibtex
@article{zhang2024categoricaljacobian,
  title   = {Protein language models learn evolutionary statistics of interacting sequence motifs},
  author  = {Zhang, Z. and Wayment-Steele, H. K. and Brixi, G. and Wang, H. and Kern, D. and Ovchinnikov, S.},
  journal = {Proceedings of the National Academy of Sciences},
  volume  = {121},
  number  = {45},
  pages   = {e2406285121},
  year    = {2024},
  doi     = {10.1073/pnas.2406285121}
}
```

## License

Apache 2.0 — see [LICENSE](https://github.com/garykbrixi/minerva/blob/main/LICENSE). Minerva-MLM is initialized from
[gLM2 650M](https://github.com/TattaBio/gLM2) (Tatta Bio, Apache 2.0).
