BioCluster

Python package for the quantitative and qualitative characterization of Pareto-optimal gene clustering solutions.

This project was designed to support the analysis of multiple clustering solutions generated by multi-objective optimization approaches, without discarding the diversity of information present across the Pareto front. The package provides utilities for reading and storing clustering results, computing structural and biological similarities, generating consensus solutions, and exporting interactive visualizations for downstream interpretation. It also supports biological characterization through Gene Ontology (GO) enrichment and GO-based network visualizations.

Statement of need

Multi-objective optimization algorithms applied to gene expression data produce a Pareto front of clustering solutions rather than a single partition. Existing bioinformatics tools, such as clusterProfiler, WGCNA, and general-purpose clustering libraries, assume a single best solution and provide no infrastructure for comparing, summarizing, or biologically interpreting an entire solution set. BioCluster fills this gap by offering an integrated Python toolkit that computes structural similarity across partitions (Jaccard, Rand, Adjusted Rand), builds consensus representations from those partitions, identifies equivalent clusters across solutions using the Hungarian algorithm, and provides GO enrichment and GO-network visualizations tuned to the multi-solution context. The primary audience is computational biologists and bioinformaticians who apply multi-objective metaheuristics (e.g., NSGA-II, MOEA/D) to transcriptomic clustering and need reproducible, publication-ready downstream analysis.

Overview

Multi-objective gene clustering commonly produces a set of non-dominated solutions rather than a single best partition. Each solution may capture different trade-offs between expression structure and biological coherence. This package helps analyze that full solution set through:

• Quantitative comparison of clustering solutions using metrics such as Jaccard similarity and Rand-based indices.
• Consensus and hierarchical clustering to summarize shared structure across multiple partitions, with automatic or manual cluster-count selection.
• Cross-solution summary analysis to quantify how much solutions agree, disagree, and which genes drive that (dis)agreement.
• Qualitative biological interpretation using GO enrichment and GO semantic relationships.
• Interactive visual outputs that help inspect similarities, cluster relationships, and enriched biological terms.

Features

Quantitative analysis

• Read clustering solutions defined over a shared gene set.
• Store and reuse matrices and tabular outputs.
• Compute solution-level and cluster-level similarity matrices.
• Compare clusterings using:
  • Jaccard similarity
  • Rand Index
  • Adjusted Rand Index
• Build coincidence / co-association representations.
• Detect equivalent clusters across solutions.

Consensus and hierarchical clustering

• Consensus-oriented workflows from similarity/coincidence structures.
• Hierarchical clustering over the consensus matrix with a manually chosen number of groups.
• Automatic cluster-count detection via the inconsistency coefficient of the linkage matrix, with a two-panel dendrogram + inconsistency-profile view comparing candidate cuts.

Summary and discrepancy analysis

• Consensus-distance scoring and outlier-solution detection relative to the consensus.
• Gene-overlap frequency analysis and frequency-cutoff selection across solutions.
• Semantic-structural discrepancy analysis and identification of the most discrepant solution pairs.

Gene Ontology analysis

• Entrez identifier mapping support.
• On-demand download of GO annotation (GAF) and NCBI gene_info reference files for several species (see GO reference data).
• GO enrichment integration workflows.
• GO semantic interpretation support.
• GO interaction network visualization.
• GO hierarchical tree visualization.
• GO enrichment summary plots (gene ratio / q-score).

Visualization

• Standard similarity heatmaps.
• Interactive clustered heatmap with linked, zoomable row/column dendrograms (HoloViews + Bokeh backend).
• Interactive HTML exports for exploratory analysis.
• Click-highlight embedding plots for cluster inspection.

Installation

Install the latest stable release from PyPI:

pip install biocluster

To install from source for development:

git clone https://github.com/BenjaminGonzalezH/ItalianEdge
cd ItalianEdge
pip install -e ".[dev]"

Dependencies

Typical dependencies include:

• numpy>=1.24,
• pandas>=2.0,
• networkx>=3.0,
• plotly>=5.0,
• matplotlib>=3.7,
• scikit-learn>=1.3,
• scipy>=1.10,
• goatools>=1.3,
• gprofiler-official>=1.0,
• mygene>=3.2,
• go3>=0.3.0,
• pyarrow>=14.0,
• holoviews>=1.19 (pulls in bokeh as a required dependency, used as the interactive clustered heatmap backend)

Input format

The package assumes:

• A shared gene universe
• Multiple clustering solutions defined over the same genes

Example:

|Gene | Sol_1 | Sol_2 | Sol_3 | |GeneA | 0 | 1 | 0 | |GeneB | 0 | 1 | 2 | |GeneC | 1 | 0 | 2 | |GeneD | 1 | 0 | 1 |

GO reference data

GO enrichment and GO-network functions need two reference files per species: a GO annotation file (.gaf) and an NCBI gene_info file. These are not bundled with the package or the repository — they must be obtained locally before running GO-related analyses (via examples/resources/ or any path of your choice).

The package can download and cache them for you:

from biocluster.go.go_utils import ensure_gaf_file, ensure_gene_info_file

gaf_path = ensure_gaf_file("tair", out_dir="examples/resources")
gene_info_path = ensure_gene_info_file("tair", out_dir="examples/resources")

Supported species_key values out of the box: goa_human (human), mgi (mouse), fb (fly), zfin (zebrafish), sgd (yeast), tair (Arabidopsis thaliana), wb (C. elegans). Files are downloaded once and reused on subsequent calls if already present in out_dir.

Quick Start

import numpy as np
from biocluster.clustering.jaccard_values import jaccard_index_solutions
from biocluster.clustering.consensus_matrix import consensus_matrix
from biocluster.visualization.heatmaps import plot_clustered_heatmap

genes = ["GeneA", "GeneB", "GeneC", "GeneD"]

solutions = np.array([
    [0, 0, 1, 1],
    [1, 1, 0, 0],
    [0, 2, 2, 1],
])

# Compute similarity
jaccard_matrix = jaccard_index_solutions(solutions)

# Build consensus
coincidence_matrix, consensus = consensus_matrix(solutions)

# Visualization
# fig = plot_clustered_heatmap(consensus, genes)

# Automatic hierarchical clustering (no fixed number of groups required)
# from biocluster.clustering.he_inconsistency_clustering import he_inconsistency_clustering
# he_inconsistency_clustering(consensus, genes, save_html_to="inconsistency.html")

# GO analysis (requires local .gaf / .obo files, see "GO reference data" above)
# from biocluster.visualization.go_network import plot_go_interaction_network_html
# plot_go_interaction_network_html(gene2terms, term_pvalues, gaf_path, obo_path)

A complete workflow includes:

1. Load clustering solutions
2. Validate gene consistency
3. Compute similarity matrices
4. Visualize structure (heatmaps)
5. Build consensus
6. Identify equivalent clusters
7. Perform GO enrichment
8. Visualize GO networks / hierarchies
9. Export results

Full examples (in examples/):

• Pipeline_documented.ipynb — recommended starting point: a guided notebook covering the theory background and the full pipeline step by step.
• Example1_File3.py / Example3_File2.py — reproducible end-to-end pipeline scripts over two different datasets/species (Arabidopsis/TAIR and human, respectively).
• Example2_Process.py — function-level validation walkthrough with hand-derived expected values for the core mathematical functions (Jaccard, Rand/ARI, consensus matrix, hierarchical and inconsistency-based clustering).

Repository structure

project_root/
│
├── src/biocluster/
│   ├── clustering/
│   │   ├── consensus_matrix.py
│   │   ├── he_clustering.py
│   │   ├── he_inconsistency_clustering.py
│   │   ├── jaccard_values.py
│   │   ├── rand_values.py
│   │   └── solutioncluster_matrix.py
│   ├── go/
│   │   ├── gene_similarity.py
│   │   ├── go_enrichment.py
│   │   ├── go_utils.py
│   │   └── mapping_entrez.py
│   ├── summary/
│   │   ├── consensus_distance.py
│   │   ├── gene_overlap.py
│   │   └── semantic_structural_discrepancy.py
│   ├── utils/
│   │   ├── actions.py
│   │   └── read_solution.py
│   └── visualization/
│       ├── go_hierarchical_network.py
│       ├── go_network.py
│       ├── go_plots.py
│       └── heatmaps.py
│
├── examples/
├── tests/
│
├── pyproject.toml
├── README.md
├── LICENSE
└── CHANGELOG.md

Outputs

Typical outputs:

1. Similarity matrices
2. Consensus matrices
3. Cluster matching tables
4. Interactive heatmaps (HTML)
5. GO enrichment tables
6. GO networks
7. GO hierarchical DAG visualizations
8. Reproducibility

To ensure reproducibility:

1. Set random_state where available
2. Keep consistent gene ordering
3. Cache intermediate matrices
4. Store outputs instead of recomputing
5. Record dependency versions

Code quality & testing

The test suite covers all clustering, go, summary, utils, and visualization modules: 273 tests, currently at ~89% line coverage.

Run the tests:

pip install -e ".[dev]"
pytest

Run the tests with coverage:

pytest --cov=biocluster --cov-report=term-missing

Linting and static analysis (also available as dev dependencies) — the codebase currently passes all three clean:

ruff check .
black --check .
mypy src/biocluster

Automated monitoring

• CI (.github/workflows/ci.yml): runs the test suite on Python 3.9–3.13 and lints (ruff, black, mypy) on every push to main and every pull request.
• Dependabot (.github/dependabot.yml): weekly PRs for vulnerable/outdated pip and GitHub Actions dependencies.
• PyPI smoke test (.github/workflows/pypi-smoke-test.yml): weekly job that installs the latest published release from PyPI and runs a minimal end-to-end check, to catch breakage caused by upstream dependency updates even when nothing changes in this repo.

Contributing

Contributions are welcome:

1. Performance improvements
2. API stabilization
3. Documentation
4. Test coverage
5. Biological validation

Guidelines:

1. Open an issue first
2. Keep changes focused
3. Include tests
4. Preserve reproducibility

Cite

@misc{biocluster,
  title        = {BioCluster},
  author       = {Inostroza Ponta, Mario and Gonzalez Hurtado, Benjamin},
  year         = {2026},
  doi          = {PLACEHOLDER — insert Zenodo DOI after registration},
  url          = {https://github.com/BenjaminGonzalezH/ItalianEdge},
  note         = {Python package for characterization of Pareto-optimal gene clustering solutions}
}

Nota: este bloque de cita debe coincidir exactamente con la lista de autores de CITATION.cff. Si Sofía Paz Lourdes Castro es coautora del proyecto, agrégala en ambos archivos (aquí y en CITATION.cff, con afiliación/ORCID); si no, esta versión ya quedó consistente entre ambos.

AI-assisted development notice

Some parts of this project were developed or refined using AI tools for:

• Code drafting
• Refactoring
• Documentation
• Test design All final decisions and validations remain the responsibility of the authors. Users should independently validate results, especially for biological interpretation.

License

This project is distributed under the MIT License.

The MIT License permits reuse, modification, and distribution, including for commercial purposes, provided that the original copyright notice and permission notice are included.

See the LICENSE file for full details.