Metadata-Version: 2.4
Name: RefgenDetector
Version: 3.0.4
Author: Mireia Marin i Ginestar
Author-email: <mireia.marin@crg.eu>
Keywords: python
Classifier: Programming Language :: Python :: 3
Classifier: Operating System :: Unix
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pysam
Requires-Dist: psutil
Requires-Dist: rich
Requires-Dist: pandas
Requires-Dist: dnspython
Requires-Dist: msgpack
Requires-Dist: numpy
Dynamic: author
Dynamic: author-email
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: keywords
Dynamic: license-file
Dynamic: requires-dist

# EGA - RefgenDetector

RefgenDetector is a bioinformatics tool that **infers the reference genome assembly** used to create aligment files (BAM/CRAM/header) and VCFs. 

## Aligment Files

It identifies major genome releases and derived assemblies across humans and multiple other species by analyzing contig names and lengths **from the header**. Benchmarking against 94 synthetic datasets achieved a 100% accuracy rate, while large-scale testing on 918,404 real-world files demonstrated 97.13% correctness, failing only when files’ headers are incomplete.

### Description

RefgenDetector is able to infer the following reference genomes:

**Primates**

👤 Homo sapiens

- hg16
- hg17
- hg18
- GRCh37
- GRCh38
- T2T

🐒 Pan troglodytes

- pantro3_0
- Pan_troglodytes-2.1

🐵 Macaca mulatta

- Mmul10
- rheMac8
- rheMac3

**Rodents**

🐭 Mus musculus

- mm7
- mm8
- mm9
- mm10
- mm39

🐀 Rattus norvegicus

- mRatBN7_2
- Rnor_6_0

**Other Mammals**

🐷 Sus scrofa

- Sscrofa10_2
- Sscrofa11_1

**Vertebrates (Non-Mammalian)**

🐟 Danio Rerio

- danRer10
- danRer11

**Invertebrates**

🪰 Drosophila Melanogaster

- dm5
- dm6

🐛 Caenorhabditis elegans

- WBcel215
- WBcel235

**Microorganisms & Plants**

🧫 Escherichia coli

- ASM886v2
- ASM584v2

🌱 Arabidopsis thaliana

- TAIR

🍺 Saccharomyces cerevisiae

- R64

## `ref_manager.py` - Customize the assemblies database. 

`ref_manager.py` provides command-line management of reference genomes used by RefgenDetector. It allows users to add custom assemblies from FASTA index (`.fai`) files, list all available references, and remove previously added custom entries without modifying the source code.

### Usage

```bash
python ref_manager.py <command> [options]
```

### Commands

#### Add a reference

```bash
python ref_manager.py add <genome.fai> <reference_name> <species> # script
refgenDetector-manager add <genome.fai> <reference_name> <species> # pip installation
```

Registers a new reference from a valid `.fai` file. If the contig structure matches an existing reference, the entry is not added.

#### List references

```bash
python ref_manager.py list # scripts
refgenDetector-manager list # pip installation
```

Displays all available references, including both built-in and user-defined assemblies.

#### Remove a reference

```bash
python ref_manager.py remove <reference_name> # scripts
refgenDetector-manager remove <reference_name> # pip installation
```

Removes a custom reference from the local database. Built-in references cannot be removed.

### Notes

- Custom references are stored separately from the default reference database.
- Input files must be valid FASTA index files generated with `samtools faidx`.
- Duplicate assemblies are detected based on exact contig composition.

## Variant Calling Files (VCFs)

From VCF files only 4 human assemblies can be inferred: 

- Hg18
- GRCh37
- GRCh38
- T2T

Two different sources of information are used to infer the reference genome from variant calling files

* **Header**
  
In the VCF specification it is recommended, but **not mandatory** that the VCF header includes tags describing the reference and contigs backing the data contained in the file. When present, the tool will analyze this information and output the reference genome version based on the contig lengths, following the same logic of the aligment files inference.

* **Variants**

To infer the reference genome from a VCF the tool will read the VCF file in chunks of 100.000 variants, avoiding to load the complete file in memory. The `POS` and `REF` columns will be extracted and compared to the msgpack files.

The msgpack files were created comparing the nucleotides in each position for hg18, GRCh37, GRCh38 and T2T. Each file contains a list of the positions where each reference had a different nucleotide (distinguishing positions). 

By getting the number of matches between these distinguishing positions and the `REF` present in the VCF we infer the reference genome version used to call the variants. 

## Requirements

- Python 3.10.6

Depending on how you want to install the package:

- pip
- Docker

## Installation

### Cloning this repository

1. Clone this repository

2.```git clone https://github.com/EGA-archive/refgenDetector.git
cd refgenDetector
pip install -e . ```

3. ``$ python3 refgenDetector_main.py -h ``

4. Download the `msgpack` files for the inference with VCFs: [Download the msgpack reference](https://crgcnag-my.sharepoint.com/:u:/g/personal/mimarin_crg_es/IQDa5CICZDAoRZmbfhBG3ZPEAWdVnNqvefFJB_r5Hc8aM70?e=kID7zn)

5. Move the `msgpack` to the correct path:

```
mv msgpacks.zip /refgenDetector/src/refgenDetector/
unzip /refgenDetector/src/refgenDetector/msgpacks.zip
```

### From pypi

``$ pip install refgenDetector``


## Usage

You can get the help menu by running:

```
$ refgenDetector -h
```

```
usage: INFERRING THE REFERENCE GENOME USED TO ALIGN BAM OR CRAM FILE [-h] -f FILE -t {BAM/CRAM,Header,VCF,BIM} [--md5] [-a] [-v MAX_N_VAR] [-m MATCHES] [-r]

optional arguments:
  -h, --help            show this help message and exit
  -f FILE, --file FILE  Input file path
  -t {BAM/CRAM,Header,VCF,BIM}, --type {BAM/CRAM,Header,VCF,BIM}
                        Type of files to analyze.
  --md5                 Print md5 values if present in header.
  -a, --assembly        Print assembly if present in header.
  -v MAX_N_VAR, --max_n_var MAX_N_VAR
                        Maximum number of variants to read before stopping inference. The file is processed in chunks of 100,000 variants, so this value must be a multiple of 100,000 (e.g. 100000,
                        200000, 300000, ...).
  -m MATCHES, --matches MATCHES
                        Number of matches required before stopping. [DEFAULT:5000]
  -r, --resources       When set, print execution time, CPU, memory, and disk I/O usage
```

## Test RefgenDetector

In the folder [examples](https://github.com/EGA-archive/refgenDetector/tree/main/examples) you can find headers, alignment and variant files to test the working of RefgenDetector.

## Licence and funding

RefgenDetector is released under GNU General Public License v3.0.

It was funded by ELIXIR, the research infrastructure for life-science data (ELIXIR Beacon Implementation Studies
2019-2021 and 2022-2023).
