Metadata-Version: 2.4
Name: feefipho
Version: 0.0.2
Summary: Functionally-informed, embedding-based phage host identification pipeline.
Project-URL: Homepage, https://huggingface.co/AlexandreBoulay/feefipho
Author: Alexandre Boulay, Clovis Galiez, Elsa Rousseau
Author-email: Alexandre Boulay <alexandre.boulay.6@ulaval.ca>
Keywords: bacteriophages,host prediction,host range,phages
Classifier: License :: OSI Approved :: GNU General Public License v3 (GPLv3)
Classifier: Programming Language :: Python :: 3
Requires-Python: >=3.11.5
Requires-Dist: biopython==1.87
Requires-Dist: empathi
Requires-Dist: huggingface-hub
Requires-Dist: joblib==1.2.0
Requires-Dist: numpy==1.26.4
Requires-Dist: pandas==2.2.1
Requires-Dist: scikit-learn==1.5.0
Requires-Dist: scipy==1.13.1
Requires-Dist: sentencepiece==0.2.0
Requires-Dist: sublyme==1.2.2
Requires-Dist: torch==2.3.0
Requires-Dist: transformers==4.43.1
Description-Content-Type: text/markdown

<h1 align="center">FEE FI PHO PHAGE<br>I smell the host of this bacteriophage</h1>

<div align="center">A <strong>F</strong>ramework <strong>E</strong>mploying protein <strong>E</strong>mbeddings for the <strong>F</strong>unctionally-<strong>I</strong>nformed <strong>P</strong>rediction of the <strong>HO</strong>sts of <strong>PHAGE</strong>s</div>

<p align="center">
  <img src="notebooks/cover.png" border="0" width="80%">
  Image generated using ChatGPT.
</p>


<!-- TABLE OF CONTENTS -->

<details open>
  <summary>Table of Contents</summary>
  <ol>
    <li><a href="#about-the-project">About the Project</a></li>
    <li><a href="#citation">Citation</a></li>
    <li>
      <a href="#getting-started">Getting Started</a>
      <ul>
        <li><a href="#prerequisites">Prerequisites</a></li>
        <li><a href="#installation">Installation</a></li>
      </ul>
    </li>
    <li><a href="#usage-details">Usage details</a></li>
    <li><a href="#output-format">Output format</a></li>
    <li><a href="#output-format">Training new models</a></li>
  </ol>
</details>

## About the project

This repo contains the code to train and launch models for the functionally-informed prediction of the bacterial host of phages based on their protein embeddings.
The classifiers are trained using the functional predictions obtained from [Empathi](https://huggingface.co/AlexandreBoulay/EmPATHi) and [SUBLYME](https://github.com/Rousseau-Team/sublyme) and use as input the protein embeddings generated using [ProtT5](https://huggingface.co/Rostlab/prot_t5_xl_half_uniref50-enc).
Independent binary models (SVMs) are trained - using only one protein type (function) at a time - to predict if a protein is associated to a host of interest or not (i.e., host1 vs all).
The predictions from each function-specific model are then combined to make a final prediction per host of interest.


## Citation

Preprint forthcoming. In the meantime please cite:

Boulay, A., Rousseau, E., Galiez, C. Feefipho. https://huggingface.co/AlexandreBoulay/feefipho.


## Getting started

Feefipho can be installed via [PyPI](https://pypi.org/project/feefipho/), [Apptainer](http://cloud.sylabs.io/library/alexandreboulay/feefipho/feefipho) or directly from the [HuggingFace repository](https://huggingface.co/AlexandreBoulay/feefipho).

A script is provided to download required models.



### Prerequisites

A GPU is recommended to compute embeddings for large datasets.

The full list of dependencies can be found in [requirements.txt](https://huggingface.co/AlexandreBoulay/feefipho/blob/main/requirements.txt).

Dependencies are taken care of by pip or Apptainer.

```
python==3.11.5
joblib==1.2.0
numpy==1.26.4
pandas==2.2.1
torch==2.3.0
scipy==1.13.1
scikit-learn==1.3.0
transformers==4.43.1
sentencepiece==0.2.0
```


### Installation

**From PyPI**:

First, create a virtual environment in python 3.11.5. For example:
```
conda create -n feefipho_env python=3.11.5
conda activate feefipho_env
```

Download Feefipho and its dependencies:
```
pip install feefipho
```

Download required models:
```
feefipho download --feefipho --empathi
```

If you plan on running the tool offline, you will also require ProtT5 models to be downloaded locally (add `--protT5` to command above). 
Note that the "Rostlab" folder must be placed in the same directory from which you will run the "feefipho predict" command.

If you are only interested in one bacterial host in particular (or in using only one type of protein), you may specify which in the download command:
```
feefipho download --feefipho Staphylococcus Mycobacterium #or
feefipho download --feefipho lysin
```

All models associated to only these bacteria or functions will be downloaded. Specifying a combination of hosts and functions will further restrict the download to this combination.


**From Apptainer**:

Download [Apptainer](https://apptainer.org/docs/admin/main/installation.html) or singularity. On windows, this will require a virtual machine.
[WSL](https://learn.microsoft.com/en-us/windows/wsl/install) works well.

Fetch the container from [GHCR](https://github.com/users/AlexandreBoulay/packages/container/package/feefipho):
```
apptainer pull oras://ghcr.io/alexandreboulay/feefipho:latest #make sure to include 'oras://' if using apptainer
```

Download required models:
```
apptainer run feefipho.sif download --feefipho --empathi  #if running offline, --protT5
```

Usage:
```
apptainer run feefipho.sif predict input.faa [options]
```


**From source**:

```
winget install git-xet
git clone https://huggingface.co/AlexandreBoulay/feefipho
cd feefipho
pip install -r requirements.txt
```

ex. 
```
python src/feefipho/download.py --empathi_models
python src/feefipho/feefipho.py test/test.faa -o test/outputs --empathi\_models empathi\_models/ -m host\_models
```

### Usage details

**Quick launch**:
```
feefipho predict input.faa -m host\_models --empathi\_models empathi\_models
```

Only make predictions for a given host or function:
```
feefipho predict input.faa -m host\_models --empathi\_models empathi\_models -b Staphylococcus
feefipho predict input.faa -m host\_models --empathi\_models empathi\_models -f lysin
```

Resume a previous run from a given checkpoint (calc\_embeddings, pred\_functions, pred\_host\_step1, pred\_host\_step2):
```
feefipho predict output/input.csv -m host\_models --empathi\_models empathi\_models -c pred\_host\_step1 #if embeddings and functional prediction is already completed. Note that the input file now corresponds to the embeddings calculated previously.
```

A fasta file of protein sequences or a csv file of protein embeddings can be used as input.
All proteins from a same phage should be separated in a similar way as phageID_proteinID (see --separator option).

Specifying the option --only\_embeddings will only compute embeddings. This step is much faster with a GPU.
The embeddings file can then be reinputted using the same command (without --only\_embeddings) and specifying the new embedding file as input file.


Options:

|Parameter|Description|
|-|-|
|**input\_file**<img width=250/>|Path to input file containing protein sequences (.fa\*) or protein embeddings (.csv) that you wish to annotate.|
|**--threads** (-t)|Number of threads (default 1).|
|**--output\_folder** (-o)|Path to the output folder. Default folder is ./host_predictions/.|
|**--only\_embeddings**|Whether to only calculate embeddings (no lysin prediction).|
|**--bacterial\_host** (-b)<img width=250/><img width=250/>|Bacterial host of interest. Default behaviour is to make predictions for all hosts. Either "all", "gram-pos", "gram-neg", "host\_type" or genus of bacterial host of interest with first letter capitalized (ex. Staphylococcus).|
|**--function** (-f)|Function of interest. By default all functions are used. Ex. "all", "lysin", etc.|
|**--models\_folder** (-m)|Path to folder containing pretrained host models. Default is ./models|
|**--empathi\_models**|Path to folder containing Empathi models.|
|**--checkpoint** (-c)<img width=250/><img width=250/>|Last checkpoint completed. Only applies if this is a continuation of a previous run that failed or was stopped. Specify one of {calc\_embeddings, pred\_functions, pred\_host\_step1, pred\_host\_step2}. If unsure see checkpoints.log.|
|**--separator** (-s)|Seperator between phageID and proteinID (ex. if phageID\_proteinID then s="\_").|
|**--separator_position**<img width=250/><img width=250/>|Describes relationship between phageID and proteinID. If "right", the split will be made on last occurrence of "separator" (ex. NIC06_P2_01), if "left", the split will be made on first occurrence of "separator" (ex. IMGVR_UViG_2529292985_000008|2529387514|2530772340).|



### Output format

When evaluating predictions, it is important to consider not only the host_proba (mean prediction over all proteins), but also the support (number of proteins that agree with this host prediction). The more proteins agree with a prediction, the more that phage ressembles known phages that infect that host, even if it also possesses many other proteins so far not known to be associated with that host.

Along with the host prediction, all functional predictions and embeddings will be outputted by the pipeline.

**final\_predictions.csv** contains the final host prediction per phage. It is formatted as follows:

|accession|host|host\_proba|coverage|support|
|-|-|-|-|-|
|AB009866|Staphylococcus|0.812|0.878|43|
|AB609718|Enterococcus|0.838|0.890|130|

where "host\_proba" corresponds to the mean of all protein-level prediction probabilities, "support" corresponds to the number of proteins that agree with the final prediction and "coverage" corresponds to the proportion of all proteins that support the final prediction (without considering proteins that do not contribute to the prediction, i.e., proteins with no predicted function).

**functional\_predictions.csv** contains the probabilities associated to the functional predictions of all proteins (Empathi and SUBLYME). A prediction >0.5 corresponds to a positive prediction.

**protein\_host\_predictions.csv** contains the probabilities of predictions made by protein-level models. For each protein, a prediction was made for each function-host-specific model (given that the protein was predicted to have that function). This file can be used to identify proteins (or types of proteins) that contributed positively to the final prediction.

**phage\_g{pos|neg}\_host\_predictions.csv** contains the probabilities of all host predictions agglomerated per phage, that can be used to create a ranking of the most probable hosts per phage.

**protein\_gram\_predictions.csv** and **phage\_gram\_predictions.csv** contains the same information but for gram-pos/gram-neg classification instead of for host genus prediction.



### Training new models

This section provides information relating to the training of models.


A binary model is trained per protein-type and per bacterial host using protein embeddings as input.


#### Requirements

Data files (found in the Zenodo archive)

* embeddings.csv - protein embeddings of all proteins.
* metadata.csv - bacterial host, protein cluster, viral cluster, etc.
* functional\_predictions.csv - Empathi and SUBLYME predictions.

Scripts

* split\_dataset.py - create train-test splits
* train.py - train models

If you want to add your own data, follow the format present in the data files. Embeddings must be generated with the ProtT5 version linked above, unless you recalculate all embeddings, or if you use only your own data.

#### Step 1. Generate splits.

Five splits are generated (20% of data in each). Four splits are used for training and the last is saved for testing. As a check of robustness, all 5 splits can be used to train/test models.

The splits are saved for the sake of reproducibility.

Protein clusters at 30% seq ID and 80% cov are used to split dataset. Proteins in the training and testing set are as distinct as possible and overfitting is limited as much as possible in each protein-level model. Taken individually, these models allow us to evaluate the predictive power of each protein-type for each host (i.e. which protein function is a better predictor for a given host).

To generate the splits:

```
python scripts/split\_train\_test\_clusters.py data/metadata.py random\_pc Escherichia -o new\_splits -f lysin --function\_file data/functional\_prediction.py
```


#### Step 2. Train models

Binary SVMs using protein embeddings as input are trained for each host independently.

```
ex. python scripts/train.py data/ random_pc lysin Staphylococcus --ratio_classes 5 -s combined -o new_models/ --data_augmentation 2
```

#### Step 3. Make predictions

You can use the main tool to make predictions. Make sure to use the newly trained models with the --models_folder (-m) option.

