Metadata-Version: 2.4
Name: biocluster
Version: 1.0.2
Summary: Toolkit for gene clustering characterization
Author-email: Mario Inostroza Ponta <mario.inostroza@usach.cl>, Benjamín González Hurtado <benjamin.gonzalez.h@usach.cl>
License-Expression: MIT
Project-URL: Homepage, https://github.com/BenjaminGonzalezH/ItalianEdge
Project-URL: Repository, https://github.com/BenjaminGonzalezH/ItalianEdge
Project-URL: Issues, https://github.com/BenjaminGonzalezH/ItalianEdge/issues
Keywords: bioinformatics,gene ontology,clustering,ensemble clustering,semantic similarity,GO visualization
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Operating System :: OS Independent
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.24
Requires-Dist: pandas>=2.0
Requires-Dist: networkx>=3.0
Requires-Dist: plotly>=5.0
Requires-Dist: matplotlib>=3.7
Requires-Dist: scikit-learn>=1.3
Requires-Dist: scipy>=1.10
Requires-Dist: goatools>=1.3
Requires-Dist: gprofiler-official>=1.0
Requires-Dist: mygene>=3.2
Requires-Dist: go3>=0.3.0
Requires-Dist: pyarrow>=14.0
Requires-Dist: holoviews>=1.23.1
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Requires-Dist: pytest-cov; extra == "dev"
Requires-Dist: black; extra == "dev"
Requires-Dist: ruff; extra == "dev"
Requires-Dist: mypy; extra == "dev"
Dynamic: license-file

# BioCluster

[![PyPI version](https://img.shields.io/pypi/v/biocluster)](https://pypi.org/project/biocluster/1.0.2/)
[![CI](https://github.com/BenjaminGonzalezH/ItalianEdge/actions/workflows/ci.yml/badge.svg)](https://github.com/BenjaminGonzalezH/ItalianEdge/actions/workflows/ci.yml)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
[![DOI](https://img.shields.io/badge/DOI-pending-lightgrey)](https://zenodo.org)

Python package for the quantitative and qualitative characterization of Pareto-optimal gene clustering solutions.

This project was designed to support the analysis of multiple clustering solutions generated by multi-objective optimization approaches, without discarding the diversity of information present across the Pareto front. The package provides utilities for reading and storing clustering results, computing structural and biological similarities, generating consensus solutions, and exporting interactive visualizations for downstream interpretation. It also supports biological characterization through Gene Ontology (GO) enrichment and GO-based network visualizations.

## Statement of need

Multi-objective optimization algorithms applied to gene expression data produce a **Pareto front of clustering solutions** rather than a single partition. Existing bioinformatics tools, such as clusterProfiler, WGCNA, and general-purpose clustering libraries, assume a single best solution and provide no infrastructure for comparing, summarizing, or biologically interpreting an entire solution set. BioCluster fills this gap by offering an integrated Python toolkit that computes structural similarity across partitions (Jaccard, Rand, Adjusted Rand), builds consensus representations from those partitions, identifies equivalent clusters across solutions using the Hungarian algorithm, and provides GO enrichment and GO-network visualizations tuned to the multi-solution context. The primary audience is computational biologists and bioinformaticians who apply multi-objective metaheuristics (e.g., NSGA-II, MOEA/D) to transcriptomic clustering and need reproducible, publication-ready downstream analysis.

## Overview

Multi-objective gene clustering commonly produces a **set of non-dominated solutions** rather than a single best partition. Each solution may capture different trade-offs between expression structure and biological coherence. This package helps analyze that full solution set through:

- **Quantitative comparison** of clustering solutions using metrics such as Jaccard similarity and Rand-based indices.
- **Consensus and hierarchical clustering** to summarize shared structure across multiple partitions, with automatic or manual cluster-count selection.
- **Cross-solution summary analysis** to quantify how much solutions agree, disagree, and which genes drive that (dis)agreement.
- **Qualitative biological interpretation** using GO enrichment and GO semantic relationships.
- **Interactive visual outputs** that help inspect similarities, cluster relationships, and enriched biological terms.

## Features

### Quantitative analysis
- Read clustering solutions defined over a shared gene set.
- Store and reuse matrices and tabular outputs.
- Compute solution-level and cluster-level similarity matrices.
- Compare clusterings using:
  - Jaccard similarity
  - Rand Index
  - Adjusted Rand Index
- Build coincidence / co-association representations.
- Detect equivalent clusters across solutions.

### Consensus and hierarchical clustering
- Consensus-oriented workflows from similarity/coincidence structures.
- Hierarchical clustering over the consensus matrix with a manually chosen number of groups.
- Automatic cluster-count detection via the inconsistency coefficient of the linkage matrix, with a two-panel dendrogram + inconsistency-profile view comparing candidate cuts.

### Summary and discrepancy analysis
- Consensus-distance scoring and outlier-solution detection relative to the consensus.
- Gene-overlap frequency analysis and frequency-cutoff selection across solutions.
- Semantic-structural discrepancy analysis and identification of the most discrepant solution pairs.

### Gene Ontology analysis
- Entrez identifier mapping support.
- On-demand download of GO annotation (GAF) and NCBI `gene_info` reference files for several species (see [GO reference data](#go-reference-data)).
- GO enrichment integration workflows.
- GO semantic interpretation support.
- GO interaction network visualization.
- GO hierarchical tree visualization.
- GO enrichment summary plots (gene ratio / q-score).

### Visualization
- Standard similarity heatmaps.
- Interactive clustered heatmap with linked, zoomable row/column dendrograms (HoloViews + Bokeh backend).
- Interactive HTML exports for exploratory analysis.
- Click-highlight embedding plots for cluster inspection.

## Installation

Install the latest stable release from PyPI:

```bash
pip install biocluster
```

To install from source for development:

```bash
git clone https://github.com/BenjaminGonzalezH/ItalianEdge
cd ItalianEdge
pip install -e ".[dev]"
```

## Dependencies
Typical dependencies include:
- numpy>=1.24,
- pandas>=2.0,
- networkx>=3.0,
- plotly>=5.0,
- matplotlib>=3.7,
- scikit-learn>=1.3,
- scipy>=1.10,
- goatools>=1.3,
- gprofiler-official>=1.0,
- mygene>=3.2,
- go3>=0.3.0,
- pyarrow>=14.0,
- holoviews>=1.23.1,
- bokeh (required by `holoviews` for the interactive clustered heatmap backend; not installed by a bare `pip install holoviews`, so make sure it's available — see note below)

> **Nota interna (no publicar sin resolver):** `bokeh` no está declarado explícitamente en `pyproject.toml`. `holoviews` por sí solo no lo instala como dependencia obligatoria, y `heatmaps.plot_clustered_heatmap` hace `from bokeh.models import HoverTool` directamente. Recomiendo agregar `"bokeh"` a `dependencies` en `pyproject.toml` antes de publicar, o esa función fallará en una instalación limpia desde PyPI.

## Input format
The package assumes:
* A shared gene universe
* Multiple clustering solutions defined over the same genes

Example:

|Gene  |   Sol_1 |  Sol_2  |  Sol_3  |
|GeneA |     0   |    1    |    0    |
|GeneB |     0   |    1    |    2    |
|GeneC |     1   |    0    |    2    |
|GeneD |     1   |    0    |    1    |

## GO reference data

GO enrichment and GO-network functions need two reference files per species: a GO annotation file (`.gaf`) and an NCBI `gene_info` file. **These are not bundled with the package or the repository** — they must be obtained locally before running GO-related analyses (via `examples/resources/` or any path of your choice).

The package can download and cache them for you:

```python
from biocluster.go.go_utils import ensure_gaf_file, ensure_gene_info_file

gaf_path = ensure_gaf_file("tair", out_dir="examples/resources")
gene_info_path = ensure_gene_info_file("tair", out_dir="examples/resources")
```

Supported `species_key` values out of the box: `goa_human` (human), `mgi` (mouse), `fb` (fly), `zfin` (zebrafish), `sgd` (yeast), `tair` (*Arabidopsis thaliana*), `wb` (*C. elegans*). Files are downloaded once and reused on subsequent calls if already present in `out_dir`.

## Quick Start

```python
import numpy as np
from biocluster.clustering.jaccard_values import jaccard_index_solutions
from biocluster.clustering.consensus_matrix import consensus_matrix
from biocluster.visualization.heatmaps import plot_clustered_heatmap

genes = ["GeneA", "GeneB", "GeneC", "GeneD"]

solutions = np.array([
    [0, 0, 1, 1],
    [1, 1, 0, 0],
    [0, 2, 2, 1],
])

# Compute similarity
jaccard_matrix = jaccard_index_solutions(solutions)

# Build consensus
coincidence_matrix, consensus = consensus_matrix(solutions)

# Visualization
# fig = plot_clustered_heatmap(consensus, genes)

# Automatic hierarchical clustering (no fixed number of groups required)
# from biocluster.clustering.he_inconsistency_clustering import he_inconsistency_clustering
# he_inconsistency_clustering(consensus, genes, save_html_to="inconsistency.html")

# GO analysis (requires local .gaf / .obo files, see "GO reference data" above)
# from biocluster.visualization.go_network import plot_go_interaction_network_html
# plot_go_interaction_network_html(gene2terms, term_pvalues, gaf_path, obo_path)
```

## A complete workflow includes:

1. Load clustering solutions
2. Validate gene consistency
3. Compute similarity matrices
4. Visualize structure (heatmaps)
5. Build consensus
6. Identify equivalent clusters
7. Perform GO enrichment
8. Visualize GO networks / hierarchies
9. Export results

Full examples (in `examples/`):
- `Pipeline_documented.ipynb` — recommended starting point: a guided notebook covering the theory background and the full pipeline step by step.
- `Example1_File3.py` / `Example3_File2.py` — reproducible end-to-end pipeline scripts over two different datasets/species (*Arabidopsis*/TAIR and human, respectively).
- `Example2_Process.py` — function-level validation walkthrough with hand-derived expected values for the core mathematical functions (Jaccard, Rand/ARI, consensus matrix, hierarchical and inconsistency-based clustering).

## Repository structure

```
project_root/
│
├── src/biocluster/
│   ├── clustering/
│   │   ├── consensus_matrix.py
│   │   ├── he_clustering.py
│   │   ├── he_inconsistency_clustering.py
│   │   ├── jaccard_values.py
│   │   ├── rand_values.py
│   │   └── solutioncluster_matrix.py
│   ├── go/
│   │   ├── gene_similarity.py
│   │   ├── go_enrichment.py
│   │   ├── go_utils.py
│   │   └── mapping_entrez.py
│   ├── summary/
│   │   ├── consensus_distance.py
│   │   ├── gene_overlap.py
│   │   └── semantic_structural_discrepancy.py
│   ├── utils/
│   │   ├── actions.py
│   │   └── read_solution.py
│   └── visualization/
│       ├── go_hierarchical_network.py
│       ├── go_network.py
│       ├── go_plots.py
│       └── heatmaps.py
│
├── examples/
├── tests/
│
├── pyproject.toml
├── README.md
├── LICENSE
└── CHANGELOG.md
```

## Outputs

Typical outputs:
1. Similarity matrices
2. Consensus matrices
3. Cluster matching tables
4. Interactive heatmaps (HTML)
5. GO enrichment tables
6. GO networks
7. GO hierarchical DAG visualizations
8. Reproducibility


To ensure reproducibility:
1. Set random_state where available
2. Keep consistent gene ordering
3. Cache intermediate matrices
4. Store outputs instead of recomputing
5. Record dependency versions

## Code quality & testing

The test suite covers all `clustering`, `go`, `summary`, `utils`, and `visualization` modules: 273 tests, currently at ~89% line coverage.

Run the tests:

```bash
pip install -e ".[dev]"
pytest
```

Run the tests with coverage:

```bash
pytest --cov=biocluster --cov-report=term-missing
```

Linting and static analysis (also available as dev dependencies) — the codebase currently passes all three clean:

```bash
ruff check .
black --check .
mypy src/biocluster
```

### Automated monitoring

- **CI** (`.github/workflows/ci.yml`): runs the test suite on Python 3.9–3.13 and lints (`ruff`, `black`, `mypy`) on every push to `main` and every pull request.
- **Dependabot** (`.github/dependabot.yml`): weekly PRs for vulnerable/outdated `pip` and GitHub Actions dependencies.
- **PyPI smoke test** (`.github/workflows/pypi-smoke-test.yml`): weekly job that installs the latest published release from PyPI and runs a minimal end-to-end check, to catch breakage caused by upstream dependency updates even when nothing changes in this repo.

## Contributing

Contributions are welcome:
1. Performance improvements
2. API stabilization
3. Documentation
4. Test coverage
5. Biological validation

Guidelines:
1. Open an issue first
2. Keep changes focused
3. Include tests
4. Preserve reproducibility

## Cite
```bibtex
@misc{biocluster,
  title        = {BioCluster},
  author       = {Inostroza Ponta, Mario and Gonzalez Hurtado, Benjamin},
  year         = {2026},
  doi          = {PLACEHOLDER — insert Zenodo DOI after registration},
  url          = {https://github.com/BenjaminGonzalezH/ItalianEdge},
  note         = {Python package for characterization of Pareto-optimal gene clustering solutions}
}
```

> **Nota:** este bloque de cita debe coincidir exactamente con la lista de autores de `CITATION.cff`. Si Sofía Paz Lourdes Castro es coautora del proyecto, agrégala en ambos archivos (aquí y en `CITATION.cff`, con afiliación/ORCID); si no, esta versión ya quedó consistente entre ambos.

## AI-assisted development notice

Some parts of this project were developed or refined using AI tools for:
* Code drafting
* Refactoring
* Documentation
* Test design
All final decisions and validations remain the responsibility of the authors. Users should independently 
validate results, especially for biological interpretation.

## License

This project is distributed under the MIT License.

The MIT License permits reuse, modification, and distribution, including for commercial purposes, provided that the original copyright
notice and permission notice are included.

See the LICENSE file for full details.
