Metadata-Version: 2.4
Name: wcl3
Version: 0.1.1
Summary: wcl3 - transcriptomic data cross-contamination eliminator
Author: Serafim Nenarokov, Martin Kolisko
Project-URL: Homepage, https://github.com/merv1n34k/wcl3
Classifier: Programming Language :: Python :: 3
Classifier: Operating System :: OS Independent
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: matplotlib>=2.0.0
Requires-Dist: numpy>=1.14.0
Requires-Dist: scipy>=0.19.0
Requires-Dist: biopython>=1.68
Requires-Dist: tqdm>=4.14.0
Requires-Dist: PyYAML>=3.11
Dynamic: license-file

# wcl3
wcl3 is the successor of WinstonCleaner, a software tool for detecting and removing cross-contaminated
contigs from assembled transcriptomes. The program uses BLAST to identify
suspicious contigs and RPKM values to sort these as either correct or
contamination.

# Requirements

To run wcl3, the following requirements must be satisfied:
* Python 3.10+
* [blast](https://blast.ncbi.nlm.nih.gov/Blast.cgi)
* [bbtools](https://jgi.doe.gov/data-and-tools/bbtools/) (`pileup.sh`)
* [bowtie2](http://bowtie-bio.sourceforge.net/bowtie2/index.shtml)

# Installation

1. Make sure that you use Python 3.10 or newer

1. Install the package:

    `pip install wcl3`

    or, with [uv](https://docs.astral.sh/uv/):

    `uv tool install wcl3`

1. Run the tool with `wcl`.

For local development:

```
uv sync
```

# Quick Start
1. Prepare the folder with input data and an empty folder for the results
1. Generate the config.yml file by running `wcl generate_config` and specify input, output paths and other options
1. Prepare the data for your dataset (Winston will BLAST your sequences and map reads to detect the coverage)

    `wcl prepare_data`

    In your output folder all necessary data will be created.

1. `wcl find_contaminations`

# Usage
## Input
The input data should be presented as a set of triads of files for each dataset.
For each dataset it is necessary to prepare:
* left reads `.fastq`
* right reads `.fastq`
* assembled transcriptome `.fasta` file

Names of the files must be in the following format:
* `NAME_1.fastq`
* `NAME_2.fastq`
* `NAME.fasta`

For example:
* `brucei_1.fastq`
* `brucei_2.fastq`
* `brucei.fasta`
* `giardia_1.fastq`
* `giardia_2.fastq`
* `giardia.fasta`

For file names only letters, digits and `_` symbols are allowed.

All the files must be placed together in one folder.

## Configuration

The dummy `settings.yml` file can be automatically generated by the command `wcl generate_config`:

```
Options:
  -h, --help            show this help message and exit
  --output_folder=OUTPUT_FOLDER
                        Where to put new settings.yml (current folder by
                        default)
```

The list of available settings:
* `winston.paths.input` &mdash; input folder with reads and contigs
* `winston.paths.output` &mdash; output folder with the results
* `winston.paths.tools.pileup_sh` &mdash; (_optional_) bbtools `pileup.sh` execution command
* `winston.paths.tools.bowtie2` &mdash; (_optional_) bowtie2 execution command
* `winston.paths.tools.bowtie2_build` &mdash; (_optional_) bowtie2-build execution command
* `winston.hits_filtering.len_ratio` &mdash; minimal `qcovhsp` for hits filtering
* `winston.hits_filtering.len_minimum` &mdash; minimal hit lenth for hits filtering
* `winston.coverage_ratio.regular` &mdash; coverage ratio for REGULAR dataset pair type
(lower values make contamination prediction more strict, less contaminations will be found)
* `winston.coverage_ratio.close` &mdash; coverage ratio for CLOSE dataset pair type
* `winston.threads.multithreading` &mdash; enable multithreading (disabling is convenient for debugging purposes)
* `winston.threads.count` &mdash; number of threads if multithreading enabled
* `winston.tools.blast.threads` &mdash; number of threads for BLAST processing
* `winston.tools.bowtie.threads` &mdash; number of threads for bowtie2 processing
* `winston.in_memory_db` &mdash; load coverage database to RAM in the beginning.
Makes contamination lookup faster, but requires decent amount of memory.

```
winston:
  in_memory_db: false

  paths:
    input: /path/to/folder/with/data/
    output: /path/to/output/folder

  hits_filtering:
    len_ratio: 70
    len_minimum: 100

  coverage_ratio:
    REGULAR: 1.1
    CLOSE: 0.04

  threads:
    multithreading:  true
    count:   8

  tools:
    blast:
      threads: 8
    bowtie:
      threads: 8
```

## Data preparation
The first step is to prepare the data for wcl3 processing.

`wcl prepare_data`

The result will be stored in the folder, specified in `winston.paths.output` option.

After the preparation the file `types.csv` can be inspected and edited.
It contains all possible combinations of dataset pairs and their types.

The default types are:
* `CLOSE` - taxonomically close organisms
* `REGULAR` - simple pair of organisms

In `types.csv` there can also be specified any amount of custom types.
Their names must be in upper case.

```
predator,prey,95.0,LEFT_EATS_RIGHT
prey,predator,95.0,RIGHT_EATS_LEFT
```

In these case coverage ratio for each custom type must be specified in `winston.coverage_ratio` section of
 `settings.yml` file:

```
...
  coverage_ratio:
    REGULAR: 1.1
    CLOSE: 0.04
    LEFT_EATS_RIGHT: 10
    RIGHT_EATS_LEFT: 0.1
...
```


## Contamination cleanup

`wcl find_contaminations`

## Output

The results will be saved in the folder, specified in `winston.paths.output` option.

For each datasets there will be the following structure of files.

* **DATASET_NAME_clean.fasta** &mdash; clean contigs
* **DATASET_NAME_deleted.fasta** &mdash; contaminated contigs
* **DATASET_NAME_suspicious_hits.csv** &mdash; all suspicious BLAST hits
* **DATASET_NAME_contamination_sources.csv** &mdash;
sources of contaminations with a following columns: source contamination dataset name, number of sequences
* **DATASET_NAME_contaminations.csv** &mdash; list of blast hits from which contaminations were detected
* **DATASET_NAME_missing_coverage.csv** &mdash; list of contig ids without a coverage


# TODO
* Logging system
* Extended unit and integration testing
* GitHub CI workflow for tests
* Export to graph format
