Metadata-Version: 2.4
Name: nose-pipeline
Version: 1.0.3
Summary: NOSE — Novel Species Identification Pipeline (Computational Systems Biology Lab @ IIT Madras)
License: MIT
Project-URL: Homepage, https://ramanlab.github.io/NOSE/
Project-URL: Repository, https://github.com/RamanLab/NOSE
Project-URL: Issues, https://github.com/RamanLab/NOSE/issues
Keywords: bioinformatics,novel species,genomics,pipeline,metagenomics,taxonomy,phylogenomics,snakemake
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: POSIX :: Linux
Classifier: Operating System :: MacOS
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: flask>=3.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: openpyxl>=3.1
Requires-Dist: click>=8.0
Provides-Extra: science
Requires-Dist: pandas>=1.5; extra == "science"
Requires-Dist: biopython>=1.80; extra == "science"
Provides-Extra: dev
Requires-Dist: pytest>=7; extra == "dev"
Requires-Dist: black; extra == "dev"
Requires-Dist: ruff; extra == "dev"
Dynamic: license-file

<div align="center">
  <img src="https://img.shields.io/badge/NOSE-Novel%20Species%20Identification-c94a1e?style=for-the-badge&labelColor=0c0e12" alt="NOSE">

  <h1>NOvel SpEcies Identification Pipeline</h1>

  <p><strong>A Modular Snakemake Toolkit for Novelty Identification and Characterization</strong></p>

  <p>
    <a href="https://github.com/RamanLab/NOSE/blob/main/LICENSE">
      <img src="https://img.shields.io/badge/License-MIT-green?style=flat-square" alt="MIT License"/>
    </a>
    <img src="https://img.shields.io/badge/Snakemake-≥7-purple?style=flat-square&logo=snakemake" alt="Snakemake"/>
    <img src="https://img.shields.io/badge/HPC-Ready-3b82f6?style=flat-square" alt="HPC Ready"/>
    <img src="https://img.shields.io/badge/Conda-Reproducible-10b981?style=flat-square" alt="Conda"/>
    <img src="https://img.shields.io/badge/Genome--Resolved-orange?style=flat-square" alt="Novel-Genome-Id"/>
  </p>

  <p>
    <a href="https://ramanlab.github.io/NOSE/"><strong>Documentation Site</strong></a>
    &nbsp;·&nbsp;
    <a href="https://github.com/RamanLab/NOSE/wiki"><strong>Wiki</strong></a>
    &nbsp;·&nbsp;
    <a href="https://github.com/RamanLab/NOSE/blob/main/Prerequisites/README.md">Setup Guide</a>
    &nbsp;·&nbsp;
    <a href="https://github.com/RamanLab/NOSE/issues">Report an Issue</a>
  </p>
</div>

<br>

> NOSE is a scalable bioinformatics pipeline designed to identify and characterize novel microbial species. By integrating quality assessment, phylogenomics, and functional mapping into a unified Snakemake workflow, NOSE ensures research is reproducible, automated, and ready for HPC environments.

---

## Table of Contents

- [Overview](#overview)
- [Pipeline Structure](#pipeline-structure)
- [Installation](#installation)
  - [Step 1 — pip install](#step-1--install-the-python-package)
  - [Step 2 — conda setup](#step-2--set-up-conda--snakemake)
  - [Step 3 — Databases](#step-3--download-databases)
  - [Step 4 — Run](#step-4--run-the-pipeline)
- [Docker](#docker)
- [Modules](#modules)
  - [Module 1: Genome Summary & Classification](#module-1-genome-summary-classification)
  - [Module 2: Overall Genome-Relatedness (OGRI)](#module-2-overall-genome-relatedness-ogri)
  - [Module 3: Phylogenetic Tree Workflow](#module-3-phylogenetic-tree-workflow)
  - [Module 4: Metagenome Mapping Workflow](#module-4-metagenome-mapping-workflow)
  - [Module 5: Functional Characterization](#module-5-functional-characterization)
  - [Module 6: Metabolic Modeling Workflow](#module-6-metabolic-modeling-workflow)
- [Authors](#authors)
- [License](#license)

---

## Overview

NOSE takes a directory of genome FASTA files and returns — with full scientific evidence — whether each genome belongs to an already-described species or represents a candidate novel organism. Each module is an independent Snakemake workflow with its own conda environments, config file, and output directory.

```
FASTA Files  →  M1: Quality Check  →  M2: Novelty Screen  →  Novel / Known
                                                                    ↓
                                                     · M3: Phylogenetic Tree Workflow
                                                     · M4: Metagenome Mapping
                                                     · M5: Functional Characterization
                                                     · M6: Metabolic Modeling
```

---

## Pipeline Structure

| Module | Name | Trigger | Key Output |
|--------|------|---------|------------|
| M1 | Genome Summary & Classification | All input genomes | `genome_summary.csv` |
| M2 | Overall Genome-Relatedness (OGRI) | M1 output | `compiled_results.csv` · `potential_novel.csv` |
| M3 | Phylogenetic ML Tree | ANI < 95% (novel) | ML tree + iTOL annotation |
| M4 | Metagenome Mapping | ANI < 95% (novel) | `final_report.csv` |
| M5 | Functional Characterization | ANI < 95% (novel) | 7 CSV outputs |
| M6 | Metabolic Modeling | ANI < 95% (novel) | SBML models + `model_summary.csv` |

<br>

<p align="center">
  <img src="figures/figure1_pipeline_overview.png" alt="NOSE Pipeline Overview" width="100%"/>
</p>

> **How to read the figure:** FASTA files enter at the top. M1 and M2 run on all genomes. The ANI < 95% gate separates known species (right branch) from novel candidates (green path). All four characterization modules (M3–M6) run only on novel candidates.

---

## Installation

> **System requirement:** NOSE runs on **Linux or macOS**. Windows users must use **WSL2** (Windows Subsystem for Linux). All bioinformatics tools are installed automatically via conda.

---

### Step 1 — Install the Python package

```bash
pip install nose-pipeline
```

This gives you four CLI commands: `nose-ui` · `nose-setup` · `nose-db` · `nose-info`

No separate `git clone` needed — the pipeline itself is bundled inside the package. The first time you run `nose-ui`, `nose-setup`, or `nose-db`, it copies itself into `~/nose-pipeline` and runs from there.

> Prefer working from source instead? `git clone https://github.com/RamanLab/NOSE.git && cd NOSE && pip install -e .` uses the cloned copy in place of the bundled one.

> **Windows users:** run this inside WSL2, not PowerShell.

---

### Step 2 — Set up conda + Snakemake

```bash
nose-setup
```

This runs automatically:
- Installs Miniconda3 (if not present)
- Adds `conda-forge` and `bioconda` channels
- Creates a `snakemake` conda environment

Or set it up manually:

```bash
conda config --add channels conda-forge
conda config --add channels bioconda
conda config --set channel_priority strict
conda create -n snakemake snakemake -y
conda activate snakemake
```

---

### Step 3 — Download databases

```bash
nose-db
```

| Database | Size | Required by |
|----------|------|-------------|
| CheckM2  | ~2 GB | Module 1 (prokaryotic) |
| GTDB-Tk  | ~66 GB | Module 1 (prokaryotic) |
| EukCC    | ~4 GB | Module 1 (eukaryotic) |
| CAT      | ~12 GB | Module 1 (eukaryotic) |

> ⚠️ **GTDB-Tk is required for Module 1 prokaryotic path** — skipping it means M1 will not run for prokaryotic genomes. Only use `--skip-gtdbtk` if you are running eukaryotic genomes exclusively.

---

### Step 4 — Run the pipeline

#### Option A — Web UI (Recommended)

```bash
nose-ui
```

Opens a browser at `http://localhost:5050`. Fill in your genome directory, configure each module, and click **Run Module**.

> **Windows users:** run `run_ui.bat` by double-clicking it, or run `nose-ui` inside WSL2.

> **Large input sets:** if you're processing a large number of genomes (or very large assemblies), prefer **Option B — Command line** below. Running directly in a terminal avoids keeping a browser tab open for long runs and makes it easier to monitor resource usage and pick up where you left off if a step is interrupted.

#### Option B — Command line

```bash
conda activate snakemake
cd Module1

# Edit config.yaml with your input/output paths, then:
bash Nose_Module1.sh
```

Repeat for each module in order: M1 → M2 → M3–M6 (M3–M6 require ANI < 95% candidates from M2).

---

### Full setup reference

📖 **[Detailed Setup Guide](https://github.com/RamanLab/NOSE/blob/main/Prerequisites/README.md)** — Anaconda installation, channel configuration, WSL2 setup for Windows

📖 **[Quick Start](https://github.com/RamanLab/NOSE/blob/main/QUICK_START.md)** — One-page cheat sheet for running all six modules

---

## Docker

NOSE provides a multi-stage Docker image that bundles Miniconda3, Snakemake, and all module conda environments into a single container. The web UI is exposed on port 5050.

> **Note:** Reference databases (GTDB-Tk ~66 GB, CheckM2 ~2 GB) are **not** included in the image — they must be downloaded separately and mounted at runtime.

### Build the image

```bash
git clone https://github.com/RamanLab/NOSE.git
cd NOSE
docker build -t nose-pipeline .
```

### Run with docker-compose (recommended)

```bash
# Set your local paths, then:
GENOMES_DIR=/path/to/genomes \
OUTPUT_DIR=/path/to/output \
DB_DIR=/path/to/databases \
docker-compose up
```

Open `http://localhost:5050` in your browser.

### Run with docker directly

```bash
docker run -p 5050:5050 \
  -v /path/to/genomes:/data/genomes \
  -v /path/to/output:/data/output \
  -v /path/to/databases:/data/databases \
  nose-pipeline
```

### Path convention inside the container

| Host path | Container path | Used by |
|-----------|---------------|---------|
| Your genome folder | `/data/genomes` | All modules — set as Genome Directory in UI |
| Your output folder | `/data/output` | All modules — set as Output Directory in UI |
| Your databases folder | `/data/databases` | Module 1 (CheckM2, GTDB-Tk) |

> When running inside Docker, always use `/data/genomes`, `/data/output`, and `/data/databases` as your paths in the web UI — not local machine paths.

---

## Modules

### Module 1: Genome Summary & Classification

📁 [`Module 1`](https://github.com/RamanLab/NOSE/tree/main/Module1) &nbsp;|&nbsp; Script: `Nose_Module1.sh`

Automated Snakemake workflow for assembly quality assessment and full taxonomic classification. A single `is_euk` flag in `config.yaml` selects the prokaryotic or eukaryotic path. Genomes passing the HQ filter are written to `genome_summary.csv` for downstream use.

**Tools:**

| Path | Tools |
|------|-------|
| Prokaryotic | QUAST · CheckM2 · GTDB-Tk |
| Eukaryotic | QUAST · EukCC · CAT |

**Quality thresholds:** Completeness ≥ 95% · Contamination ≤ 5% · Classified to genus level minimum

**Outputs:**
- `genome_summary.csv` — genomes passing QC thresholds → input for Module 2
- `unqualified_genome_summary.csv` — genomes that did not meet thresholds

---

### Module 2: Overall Genome-Relatedness (OGRI)

📁 [`Module 2`](https://github.com/RamanLab/NOSE/tree/main/Module2) &nbsp;|&nbsp; Script: `Nose_Module2.sh`

Computes three complementary genomic distance metrics against all RefSeq type strains in the genome's genus, downloaded automatically via the NCBI Datasets API. Supports both WGS and 16S-only input modes. Genomes with ANI < 95% are flagged as candidate novel species.

> ⚠️ An NCBI API key must be set in `config.yaml`. Without it, reference genome downloads will be rate-limited.
> 🔗 How to get an NCBI API key: [NCBI API Integration Guide](https://github.com/Novel-sp/Genome-related-indices/blob/main/ncbi_readme.md)

**Tools:** FastANI · AAI (`aai.rb`) · POCP (`pocp.sh`) · Barrnap · BLASTn · NCBI Datasets API

**Three OGRI metrics:**

| Metric | Boundary | Significance |
|--------|----------|-------------|
| ANI | < 95% = novel | Primary species-level boundary (IJSEM standard) |
| AAI | — | Broader evolutionary distances at proteome level |
| POCP | < 50% = new genus | Genus-level delineation |

**Outputs:**
- `compiled_results.csv` — all genomes with ANI / AAI / POCP / 16S identity
- `potential_novel.csv` — ANI < 95% candidates, staged for M3–M6

<p align="center">
  <img src="figures/figure2_m1_m2_detail.png" alt="Module 1 and Module 2 detailed workflow" width="100%"/>
</p>

---

### Module 3: Phylogenetic Tree Workflow

📁 [`Module 3`](https://github.com/RamanLab/NOSE/tree/main/Module3) &nbsp;|&nbsp; Script: `Nose_Module3.sh`

Validates taxonomic novelty via genus-specific Maximum Likelihood (ML) trees built from concatenated Single-Copy Genes (SCGs). GToTree identifies SCGs using HMMs; IQ-TREE infers the ML tree with 1000 ultrafast bootstrap replicates; `tree_annotation.py` generates iTOL-ready annotation files.

> ⚠️ Requires manual addition of `Outroup` (GCF ID) and `HMM` columns to `genome_summary_mod.csv` before running.

**Tools:** GToTree · IQ-TREE · iTOL · `tree_annotation.py`

**IQ-TREE command:**
```bash
iqtree \
  -s Aligned_SCGs.faa \   # concatenated SCG alignment
  -spp Partitions.txt \   # per-gene partition model
  -m MFP \                # ModelFinder Plus
  -bb 1000 \              # ultrafast bootstraps
  -nt 4 \                 # CPU threads
  -pre {genus}_iqtree_out
```

**Outputs:** Per-genus ML tree files + iTOL annotation CSV files

<p align="center">
  <img src="figures/figure3_m3_phylogenetics.png" alt="Module 3 Phylogenetic Tree Workflow" width="90%"/>
</p>

---

### Module 4: Metagenome Mapping Workflow

📁 [`Module 4`](https://github.com/RamanLab/NOSE/tree/main/Module4) &nbsp;|&nbsp; Script: `Nose_Module4.sh`

Quantifies isolate prevalence and relative abundance across metagenomic datasets using sylph k-mer containment estimation. No BAM files or read alignment required. Species-level threshold (c=100) with minimum 5 k-mers enforced to suppress false positives.

**Tools:** sylph · pandas · `merge_results.py`

**Containment thresholds:**

| Threshold (c) | Resolution |
|---------------|------------|
| 100 | Species-level (default) |
| 95 | Genus-level |
| 90 | Family-level |

**Output:** `final_report.csv` — Sample_ID · Genome · Containment · ANI · Reads_Queried · Reads_Matching

---

### Module 5: Functional Characterization

📁 [`Module 5`](https://github.com/RamanLab/NOSE/tree/main/Module5) &nbsp;|&nbsp; Script: `Nose_Module5.sh`

Multi-modal pipeline integrating structural annotation, COG functional classification, biosynthetic gene cluster detection, resistance and virulence screening, and mobile element identification.

**Tools:** Prokka · COGclassifier · antiSMASH · ABRICATE · geNomad

**Workflow steps:**

| Step | Tool | Output |
|------|------|--------|
| Gene annotation | Prokka | `.gff` · `.faa` · `.ffn` · `.gbk` |
| Functional classification | COGclassifier | `merged_classifier_count.csv` |
| BGC detection | antiSMASH | `AntiSMASH_results.csv` |
| Resistance & virulence | ABRICATE (CARD · VFDB · BacMet · NCBI) | Per-database tabular reports |
| Phage & plasmid detection | geNomad | `virus_summary.csv` · `plasmid_summary.csv` |

> 💡 geNomad tells you whether resistance or virulence genes reside on mobile elements — meaning they can spread horizontally to other organisms.

---

### Module 6: Metabolic Modeling Workflow

📁 [`Module 6`](https://github.com/RamanLab/NOSE/tree/main/Module6) &nbsp;|&nbsp; Script: `Nose_Module6.sh`

Genome-scale metabolic reconstruction using CarveMe (top-down approach). Each GEM is validated with COBRApy unconstrained growth tests and benchmarked with MEMOTE for stoichiometric consistency. Models are exported in SBML/FBC format compatible with COBRA Toolbox, cobrapy, and OptFlux.

**Tools:** CarveMe · COBRApy · MEMOTE · `generate_model_stats.py` · `compile_model_summary.py`

> ⚠️ Always use the `--prodigal` flag with CarveMe. Default gene prediction produces incomplete GEM reconstruction for certain isolates.

**Output:** `model_summary.csv` — sample · n_reactions · n_metabolites · n_genes · growth · memote_score

<p align="center">
  <img src="figures/figure4_m4_m5_m6_characterization.png" alt="Modules 4, 5 and 6 Characterization Suite" width="100%"/>
</p>

---

## Authors

Developed at **[Computational Biology Lab, IIT Madras](https://github.com/RamanLab)**

| Role | Name |
|------|------|
| Principal Investigator | [Prof. Karthik Raman](https://github.com/ramanlab) |
| Pipeline Authors | [Prithvi](https://github.com/Prithvi-0805) · [Harippriya](https://github.com/Harippriyasiva) · [Enos](https://github.com/EnosJad) · [Pratyay Sengupta](https://github.com/pratyaysengupta) |

---

## License

This project is licensed under the **MIT License** — see the [LICENSE](https://github.com/RamanLab/NOSE/blob/main/LICENSE) file for details.

---

<p align="center">
  <sub>© 2026 NOSE Project Team · Computational Biology Lab · IIT Madras · Built for Novel Species Identification</sub>
</p>
