Metadata-Version: 2.4
Name: nose-pipeline
Version: 1.0.9
Summary: NOSE — Novel Species Identification Pipeline (Computational Systems Biology Lab @ IIT Madras)
Author: Prithvi S Prabhu, Harippriya Sivakumar, Enos Jadlin, Pratyay Sengupta, Karthik Raman
License: MIT
Project-URL: Homepage, https://ramanlab.github.io/NOSE/
Project-URL: Repository, https://github.com/RamanLab/NOSE
Project-URL: Issues, https://github.com/RamanLab/NOSE/issues
Keywords: bioinformatics,novel species,genomics,pipeline,metagenomics,taxonomy,phylogenomics,snakemake
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: POSIX :: Linux
Classifier: Operating System :: MacOS
Classifier: Operating System :: Microsoft :: Windows
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: flask>=3.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: openpyxl>=3.1
Requires-Dist: click>=8.0
Requires-Dist: waitress>=2.1
Provides-Extra: science
Requires-Dist: pandas>=1.5; extra == "science"
Requires-Dist: biopython>=1.80; extra == "science"
Provides-Extra: dev
Requires-Dist: pytest>=7; extra == "dev"
Requires-Dist: black; extra == "dev"
Requires-Dist: ruff; extra == "dev"
Dynamic: license-file

<div align="center">
  <img src="https://img.shields.io/badge/NOSE-Novel%20Species%20Identification-c94a1e?style=for-the-badge&labelColor=0c0e12" alt="NOSE">

  <h1>NOvel SpEcies Identification Pipeline</h1>

  <p><strong>A Modular Snakemake Toolkit for Novelty Identification and Characterization</strong></p>

  <p>
    <a href="https://pypi.org/project/nose-pipeline/">
      <img src="https://img.shields.io/pypi/v/nose-pipeline?style=flat-square&color=blue" alt="PyPI version"/>
    </a>
    <a href="https://pepy.tech/project/nose-pipeline">
      <img src="https://static.pepy.tech/badge/nose-pipeline" alt="PyPI downloads"/>
    </a>
    <a href="https://github.com/RamanLab/NOSE/blob/main/LICENSE">
      <img src="https://img.shields.io/badge/License-MIT-green?style=flat-square" alt="MIT License"/>
    </a>
    <img src="https://img.shields.io/badge/Snakemake-≥7-purple?style=flat-square&logo=snakemake" alt="Snakemake"/>
    <img src="https://img.shields.io/badge/HPC-Ready-3b82f6?style=flat-square" alt="HPC Ready"/>
    <img src="https://img.shields.io/badge/Conda-Reproducible-10b981?style=flat-square" alt="Conda"/>
    <img src="https://img.shields.io/badge/Genome--Resolved-orange?style=flat-square" alt="Novel-Genome-Id"/>
  </p>

  <p>
    <img src="https://visitor-badge.laobi.icu/badge?page_id=RamanLab.NOSE" alt="Visitor count"/>
  </p>

  <p>
    <a href="https://ramanlab.github.io/NOSE/"><strong>Documentation Site</strong></a>
    &nbsp;·&nbsp;
    <a href="https://github.com/RamanLab/NOSE/wiki"><strong>Wiki</strong></a>
    &nbsp;·&nbsp;
    <a href="https://github.com/RamanLab/NOSE/blob/main/Prerequisites/README.md">Setup Guide</a>
    &nbsp;·&nbsp;
    <a href="https://github.com/RamanLab/NOSE/issues">Report an Issue</a>
  </p>
</div>

<br>

> NOSE is a scalable bioinformatics pipeline designed to identify and characterize novel microbial species. By integrating quality assessment, phylogenomics, and functional mapping into a unified Snakemake workflow, NOSE ensures research is reproducible, automated, and ready for HPC environments.

---

## Table of Contents

- [Overview](#overview)
- [Pipeline Structure](#pipeline-structure)
- [Installation](#installation)
  - [Step 1 - pip install](#step-1---install-the-python-package)
  - [Step 2 - conda setup](#step-2---set-up-your-environment)
  - [Step 3 - Databases](#step-3---download-databases)
  - [Step 4 - Run](#step-4---run-the-pipeline)
- [Docker](#docker)
- [Modules](#modules)
  - [Module 1: Genome Summary & Classification](#module-1-genome-summary--classification)
  - [Module 2: Overall Genome-Relatedness (OGRI)](#module-2-overall-genome-relatedness-ogri)
  - [Module 3: Phylogenetic Tree Workflow](#module-3-phylogenetic-tree-workflow)
  - [Module 4: Metagenome Mapping Workflow](#module-4-metagenome-mapping-workflow)
  - [Module 5: Functional Characterization](#module-5-functional-characterization)
  - [Module 6: Metabolic Modeling Workflow](#module-6-metabolic-modeling-workflow)
- [Authors](#authors)
- [License](#license)

---

## Overview

The vast majority of microbial life remains uncultured and uncharacterized, and confirming that a genome represents a genuinely novel species has traditionally meant coordinating a dozen separate tools by hand - a process prone to inconsistency and difficult to reproduce. NOSE automates that entire judgment call: point it at a directory of genome FASTA files, and it returns, with full polyphasic evidence (ANI, AAI, POCP, and phylogenetic placement), whether each genome belongs to an already-described species or is a candidate for a new one - then characterizes what that novel organism can actually do.

Every module is an independent Snakemake workflow with its own conda environments, config file, and output directory, so the pipeline scales from a laptop to an HPC cluster without any code changes.

**Why NOSE:**
- **End-to-end** - from raw assemblies to phylogenetic placement, environmental prevalence, functional annotation, and metabolic models, in one run
- **Evidence-based** - the novelty call rests on three independent genomic distance metrics (ANI, AAI, POCP) plus a maximum-likelihood phylogenetic tree, not a single threshold
- **Reproducible** - every tool runs in its own version-pinned conda environment via Snakemake's `--use-conda`, with checkpointing and automatic resume on interrupted runs
- **No command line required** - the built-in web UI configures and runs every module from a browser; `pip install nose-pipeline` is the only setup step

```
FASTA Files  →  M1: Quality Check  →  M2: Novelty Screen  →  Novel / Known
                                                                    ↓
                                                     · M3: Phylogenetic Tree Workflow
                                                     · M4: Metagenome Mapping
                                                     · M5: Functional Characterization
                                                     · M6: Metabolic Modeling
```

---

## Pipeline Structure

| Module | Name | Trigger | Key Output |
|--------|------|---------|------------|
| M1 | Genome Summary & Classification | All input genomes | `genome_summary.csv` |
| M2 | Overall Genome-Relatedness (OGRI) | M1 output | `compiled_results.csv` · `potential_novel.csv` · `genome_summary_for_tree.csv` |
| M3 | Phylogenetic ML Tree | M2 output  | ML tree + iTOL annotation |
| M4 | Metagenome Mapping | M2 output | `final_report.csv` |
| M5 | Functional Characterization | M2 output | `functional_summary.csv` |
| M6 | Metabolic Modeling | M2 output | SBML models + `model_summary.csv` |

<br>

<p align="center">
  <img src="figures/figure1_pipeline_overview.png" alt="NOSE Pipeline Overview" width="100%"/>
</p>

> **How to read the figure:** FASTA files enter at the top. M1 and M2 run on all genomes. The ANI < 95% gate separates known species (right branch) from novel candidates (green path). All four characterization modules (M3–M6) run only on novel candidates.

---

## Installation

> **System requirement:** The bioinformatics tools themselves need **Linux or macOS**. On Windows, `pip install` and the web UI (`nose-ui`) work natively in PowerShell - the dashboard automatically routes actual module runs through **WSL2** (a distro named `Ubuntu` must be installed). `nose-setup` and `nose-db` need a real `bash` on PATH (WSL2 or Git Bash), so run those two specifically inside WSL2. All bioinformatics tools are installed automatically via conda.

---

### Step 1 - Install the Python package

```bash
pip install nose-pipeline
```

Latest release: **[nose-pipeline on PyPI](https://pypi.org/project/nose-pipeline/)** (currently v1.0.6 - see the badge at the top of this page, which always shows the exact version actually published).

This is the only setup step - no `git clone` needed. The package carries its own copy
of the pipeline code and unpacks it to `~/nose-pipeline` the first time you run any
command below, refreshing that copy on every launch so it always matches whatever
version you have installed. Gives you four CLI commands: `nose-ui` · `nose-setup` ·
`nose-db` · `nose-info`.

> **Windows users:** this step and `nose-ui` work fine directly in PowerShell. `nose-setup` and `nose-db` (next two steps) need WSL2, since they run bash scripts.

> **Contributing to NOSE instead of just running it?** Clone the repo directly -
> `git clone https://github.com/RamanLab/NOSE.git` - and run everything from inside
> that checkout (`pip install -e .`) instead of a plain `pip install`. `cli.py` prefers
> a real git checkout over its own bundled copy whenever one is available, so you get
> `git log`/`git pull`/`git blame` on the actual pipeline code you're editing.

---

### Step 2 - Set up your environment

```bash
nose-setup
```

Works with whichever of **conda**, **mamba**, or **micromamba** you already
have installed - it doesn't require conda specifically. If none of the
three are found, it asks before installing Miniconda3 (never silently).

It then asks what to call the environment (Enter for the default, `nose`),
confirms before creating anything, and - unlike earlier versions - verifies
the environment actually exists afterward instead of just trusting a zero
exit code, so a partial/failed create can't silently report success.

```bash
nose-setup                     # asks for a name, confirms, then creates it
nose-setup --env-name myenv    # skip the name prompt
nose-setup --yes               # no prompts at all -- scripts/CI
```

> **Windows users:** run this inside WSL2 - it shells out to a bash script directly, unlike `nose-ui`.

Or set it up manually with any of the three tools, e.g.:

```bash
conda config --add channels conda-forge
conda config --add channels bioconda
conda config --set channel_priority strict
conda create -n nose snakemake -y
conda activate nose
```

---

### Step 3 - Download databases

```bash
nose-db
```

Asks "Download now? [y/N]" before each one (unless you pass `--all`), and
checks free disk space at the destination first so you're not surprised
partway through a 70 GB download.

| Database | Size | Required by |
|----------|------|-------------|
| CheckM2  | ~2 GB | Module 1 (prokaryotic) |
| GTDB-Tk  | ~66 GB | Module 1 (prokaryotic) |
| EukCC    | ~4 GB | Module 1 (eukaryotic) |
| CAT      | ~70 GB | Module 1 (eukaryotic) |

> **GTDB-Tk is required for Module 1 prokaryotic path** - skipping it means M1 will not run for prokaryotic genomes. Only use `--skip-gtdbtk` if you are running eukaryotic genomes exclusively.

---

### Step 4 - Run the pipeline

#### Option A - Web UI (Recommended)

```bash
nose-ui
```

Opens a browser at `http://localhost:5050`. Fill in your genome directory, configure each module, and click **Run Module**.

> **Windows users:** `nose-ui` (or double-clicking `run_ui.bat`) runs directly in PowerShell - no WSL2 needed just to launch the dashboard. When you click **Run** on a module, it automatically routes execution through WSL2 (requires a WSL distro named `Ubuntu` to be installed).

> **Large input sets:** if you're processing a large number of genomes (or very large assemblies), prefer **Option B - Command line** below. Running directly in a terminal avoids keeping a browser tab open for long runs and makes it easier to monitor resource usage and pick up where you left off if a step is interrupted.

#### Option B - Command line

```bash
conda activate snakemake
cd Module1

# Edit config.yaml with your input/output paths, then:
bash Nose_Module1.sh
```

Repeat for each module in order: M1 → M2 → M3–M6 (M3–M6 require ANI < 95% candidates from M2).

---

### Full setup reference

**[Detailed Setup Guide](https://github.com/RamanLab/NOSE/blob/main/Prerequisites/README.md)** - Anaconda installation, channel configuration, WSL2 setup for Windows

**[Quick Start](https://github.com/RamanLab/NOSE/blob/main/QUICK_START.md)** - One-page cheat sheet for running all six modules

---

## Docker

NOSE provides a multi-stage Docker image that bundles Miniconda3, Snakemake, and all module conda environments into a single container. The web UI is exposed on port 5050.

> **Note:** Reference databases (GTDB-Tk ~66 GB, CheckM2 ~2 GB) are **not** included in the image - they must be downloaded separately and mounted at runtime.

### Build the image

```bash
git clone https://github.com/RamanLab/NOSE.git
cd NOSE
docker build -t nose-pipeline .
```

### Run with docker-compose (recommended)

```bash
# Set your local paths, then:
GENOMES_DIR=/path/to/genomes \
OUTPUT_DIR=/path/to/output \
DB_DIR=/path/to/databases \
docker-compose up
```

Open `http://localhost:5050` in your browser.

### Run with docker directly

```bash
docker run -p 5050:5050 \
  -v /path/to/genomes:/data/genomes \
  -v /path/to/output:/data/output \
  -v /path/to/databases:/data/databases \
  nose-pipeline
```

### Path convention inside the container

| Host path | Container path | Used by |
|-----------|---------------|---------|
| Your genome folder | `/data/genomes` | All modules - set as Genome Directory in UI |
| Your output folder | `/data/output` | All modules - set as Output Directory in UI |
| Your databases folder | `/data/databases` | Module 1 (CheckM2, GTDB-Tk) |

> When running inside Docker, always use `/data/genomes`, `/data/output`, and `/data/databases` as your paths in the web UI - not local machine paths.

---

## Modules

### Module 1: Genome Summary & Classification

[`Module 1`](https://github.com/RamanLab/NOSE/tree/main/Module1) &nbsp;|&nbsp; Script: `Nose_Module1.sh`

Automated Snakemake workflow for assembly quality assessment and full taxonomic classification. A single `is_euk` flag in `config.yaml` selects the prokaryotic or eukaryotic path. Genomes passing the HQ filter are written to `genome_summary.csv` for downstream use.

**Tools:**

| Path | Tools |
|------|-------|
| Prokaryotic | QUAST · CheckM2 · GTDB-Tk |
| Eukaryotic | QUAST · EukCC · CAT |

**Quality thresholds:** Completeness ≥ 95% · Contamination ≤ 5% · Classified to genus level minimum

**Outputs:**
- `genome_summary.csv` - genomes passing QC thresholds → input for Module 2
- `unqualified_genome_summary.csv` - genomes that did not meet thresholds

---

### Module 2: Overall Genome-Relatedness (OGRI)

[`Module 2`](https://github.com/RamanLab/NOSE/tree/main/Module2) &nbsp;|&nbsp; Script: `Nose_Module2.sh`

Computes three complementary genomic distance metrics against all RefSeq type strains in the genome's genus, downloaded automatically via the NCBI Datasets API. Supports both WGS and 16S-only input modes. Genomes with ANI < 95% are flagged as candidate novel species.

> An NCBI API key must be set in `config.yaml`. Without it, reference genome downloads will be rate-limited.
> How to get an NCBI API key: [NCBI API Integration Guide](https://github.com/RamanLab/NOSE/blob/main/Module2/ncbi_readme.md)

**Tools:** FastANI · AAI (`aai.rb`) · POCP (`pocp.sh`) · Barrnap · BLASTn · NCBI Datasets API

**Three OGRI metrics:**

| Metric | Boundary | Significance |
|--------|----------|-------------|
| ANI | < 95% = novel | Primary species-level boundary (IJSEM standard) |
| AAI | - | Broader evolutionary distances at proteome level |
| POCP | < 50% = new genus | Genus-level delineation |

**Outputs:**
- `compiled_results.csv` - all genomes with ANI / AAI / POCP / 16S identity
- `potential_novel.csv` - ANI < 95% candidates, staged for M3–M6

<p align="center">
  <img src="figures/figure2_m1_m2_detail.png" alt="Module 1 and Module 2 detailed workflow" width="100%"/>
</p>

---

### Module 3: Phylogenetic Tree Workflow

[`Module 3`](https://github.com/RamanLab/NOSE/tree/main/Module3) &nbsp;|&nbsp; Script: `Nose_Module3.sh`

Validates taxonomic novelty via genus-specific Maximum Likelihood (ML) trees built from concatenated Single-Copy Genes (SCGs). GToTree identifies SCGs using HMMs; IQ-TREE infers the ML tree with 1000 ultrafast bootstrap replicates; `tree_annotation.py` generates iTOL-ready annotation files.

> Genus, outgroup accession, and HMM set are all resolved automatically by Module 2 - no manual file preparation required. See [Module 3's README](Module3/README.md#3-how-genus-grouping-hmm-sets-and-outgroups-are-determined) for how it works, or to override a genus's automatic pick.

**Tools:** GToTree · IQ-TREE · iTOL · `tree_annotation.py`

**IQ-TREE command:**
```bash
iqtree \
  -s Aligned_SCGs.faa \   # concatenated SCG alignment
  -spp Partitions.txt \   # per-gene partition model
  -m MFP \                # ModelFinder Plus
  -bb 1000 \              # ultrafast bootstraps
  -nt 4 \                 # CPU threads
  -pre {genus}_iqtree_out
```

**Outputs:** Per-genus ML tree files + iTOL annotation CSV files

<p align="center">
  <img src="figures/figure3_m3_phylogenetics.png" alt="Module 3 Phylogenetic Tree Workflow" width="90%"/>
</p>

---

### Module 4: Metagenome Mapping Workflow

[`Module 4`](https://github.com/RamanLab/NOSE/tree/main/Module4) &nbsp;|&nbsp; Script: `Nose_Module4.sh`

Quantifies isolate prevalence and relative abundance across metagenomic datasets using sylph k-mer containment estimation. No BAM files or read alignment required. Species-level threshold (c=100) with minimum 5 k-mers enforced to suppress false positives.

**Tools:** sylph · pandas · `merge_results.py`

**Containment thresholds:**

| Threshold (c) | Resolution |
|---------------|------------|
| 100 | Species-level (default) |
| 95 | Genus-level |
| 90 | Family-level |

**Output:** `final_report.csv` - Sample_ID · Genome · Containment · ANI · Reads_Queried · Reads_Matching

---

### Module 5: Functional Characterization

[`Module 5`](https://github.com/RamanLab/NOSE/tree/main/Module5) &nbsp;|&nbsp; Script: `Nose_Module5.sh`

Multi-modal pipeline integrating structural annotation, COG functional classification, biosynthetic gene cluster detection, resistance and virulence screening, and mobile element identification.

**Tools:** Prokka · COGclassifier · antiSMASH · ABRICATE · geNomad

**Workflow steps:**

| Step | Tool | Output |
|------|------|--------|
| Gene annotation | Prokka | `.gff` · `.faa` · `.ffn` · `.gbk` |
| Functional classification | COGclassifier | `merged_classifier_count.csv` |
| BGC detection | antiSMASH | `AntiSMASH_results.csv` |
| Resistance & virulence | ABRICATE (CARD · VFDB · BacMet · NCBI) | Per-database tabular reports |
| Phage & plasmid detection | geNomad | `virus_summary.csv` · `plasmid_summary.csv` |

> geNomad tells you whether resistance or virulence genes reside on mobile elements - meaning they can spread horizontally to other organisms.

---

### Module 6: Metabolic Modeling Workflow

[`Module 6`](https://github.com/RamanLab/NOSE/tree/main/Module6) &nbsp;|&nbsp; Script: `Nose_Module6.sh`

Genome-scale metabolic reconstruction using CarveMe (top-down approach). Each GEM is validated with COBRApy unconstrained growth tests and benchmarked with MEMOTE for stoichiometric consistency. Models are exported in SBML/FBC format compatible with COBRA Toolbox, cobrapy, and OptFlux.

**Tools:** CarveMe · COBRApy · MEMOTE · `generate_model_stats.py` · `compile_model_summary.py`

> Always use the `--prodigal` flag with CarveMe. Default gene prediction produces incomplete GEM reconstruction for certain isolates.

**Output:** `model_summary.csv` - sample · n_reactions · n_metabolites · n_genes · growth · memote_score

<p align="center">
  <img src="figures/figure4_m4_m5_m6_characterization.png" alt="Modules 4, 5 and 6 Characterization Suite" width="100%"/>
</p>

---

## Authors

Developed at **[Computational Systems Biology Lab @ IIT Madras](https://github.com/RamanLab)**

| Role | Name |
|------|------|
| Principal Investigator | [Prof. Karthik Raman](https://github.com/ramanlab) |
| Pipeline Authors | [Prithvi](https://github.com/Prithvi-0805) · [Harippriya](https://github.com/Harippriyasiva) · [Enos](https://github.com/EnosJad) · [Pratyay Sengupta](https://github.com/pratyaysengupta) |

---

## License

This project is licensed under the **MIT License** - see the [LICENSE](https://github.com/RamanLab/NOSE/blob/main/LICENSE) file for details.

---

<p align="center">
  <sub>© 2026 NOSE Project Team ·  Systems Biology Lab · IIT Madras · Built for Novel Species Identification</sub>
</p>
