Metadata-Version: 2.4
Name: climb-astro
Version: 0.3.1
Summary: A versatile two-phase clustering algorithm designed for datasets with both known and exploratory components.
Author-email: Lorenzo Monti <lorenzomonti42@gmail.com>
License-Expression: MIT
Project-URL: Homepage, https://github.com/LorenzoMonti/CLiMB
Project-URL: Documentation, https://climb-astro.readthedocs.io/en/latest/
Project-URL: Repository, https://github.com/LorenzoMonti/CLiMB.git
Project-URL: BugTracker, https://github.com/LorenzoMonti/CLiMB/issues
Classifier: Programming Language :: Python :: 3
Classifier: Operating System :: OS Independent
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Requires-Python: >=3.7
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.19.0
Requires-Dist: matplotlib>=3.3.0
Requires-Dist: scikit-learn>=0.24.0
Requires-Dist: scipy>=1.6.0
Requires-Dist: pandas>=1.5.3
Requires-Dist: hdbscan>=0.8.27
Provides-Extra: dev
Requires-Dist: pytest>=6.0; extra == "dev"
Requires-Dist: black; extra == "dev"
Requires-Dist: flake8; extra == "dev"
Requires-Dist: sphinx; extra == "dev"
Requires-Dist: sphinx-rtd-theme; extra == "dev"
Dynamic: license-file

<p align="center"> <img src="imgs/icon.png" alt="CLiMB logo" width="300"/> </p>

# CLustering In Multiphase Boundaries (CLiMB)
A versatile two-phase clustering algorithm designed for datasets with both known and exploratory components.

## Features

- **Two-Phase Clustering**: Combines constrained clustering with exploratory clustering to identify both known and novel patterns.
- **Density-Aware**: Uses local density estimation to intelligently filter and assign points.
- **Flexible Exploratory Phase**: Supports multiple clustering algorithms (DBSCAN, HDBSCAN, OPTICS) through a strategy pattern.
- **Visualization Tools**: Built-in 2D and 3D visualization capabilities for cluster analysis.
- **Parameter Tuning**: Builder pattern for flexible parameter adjustment.
- **Customizable Distance Metrics**: Now supports various distance metrics such as Euclidean, Mahalanobis, and custom metrics, offering greater flexibility in distance calculation.
- **Interpretability**: `explain()` reconstructs both phases' decisions in closed form and
  verifies the reconstruction against the model, rather than fitting a surrogate to imitate it.
- **Advanced Seed Points**: Ability to initialize clustering with known seed points provided in a dictionary structure, allowing for more precise control over centroid initialization.

## Installation

```bash
pip install climb-astro
```

Or install from source:

```bash
git clone https://github.com/LorenzoMonti/CLiMB.git
cd CLiMB
pip install -e .
```

## Quick Start

```python
import numpy as np
from sklearn.datasets import make_blobs
from sklearn.preprocessing import StandardScaler
from CLiMB.core.CLiMB import CLiMB
from CLiMB.exploratory.DBSCANExploratory import DBSCANExploratory

# The number of centers to generate
centers = 4

# Generate synthetic data with 5 dimensions
X, y = make_blobs(n_samples=500, centers=centers, n_features=5, random_state=42)

# Scale the data
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

# Create seed points (optional)
seed_points = np.array([
    X[y == i].mean(axis=0) for i in range(centers)
])
seed_points_scaled = scaler.transform(seed_points)

# Example of seed points as a dictionary for more precise control
seed_dict_scaled = {
    tuple(seed_points_scaled[0]): [tuple(X_scaled[y == 0][0]), tuple(X_scaled[y == 0][1])], # Centroid 1 and associated seed points
    tuple(seed_points_scaled[1]): [tuple(X_scaled[y == 1][0])],                             # Centroid 2 and associated seed points
    tuple(seed_points_scaled[2]): [],                                                         # Centroid 3 with no specific seed points
    tuple(seed_points_scaled[3]): [tuple(X_scaled[y == 3][0]), tuple(X_scaled[y == 3][1]), tuple(X_scaled[y == 3][2])] # Centroid 4 and seed points
}

# Initialize and fit CLiMB with Euclidean metric and dictionary seed points
climb = CLiMB(
    constrained_clusters=4,
    seed_points=seed_dict_scaled, # Use the dictionary of seed points
    density_threshold=0.15,
    distance_threshold=2.5,
    radial_threshold=1.2,
    convergence_tolerance=0.05,
    distance_metric='euclidean',
    metric_params=None,
    exploratory_algorithm=DBSCANExploratory(0.5)
)
climb.fit(X_scaled)

# Get cluster labels
labels = climb.get_labels()

# Visualize results (only possible in lower dimensions)
climb.inverse_transform(scaler)
fig = climb.plot_comprehensive_3d(save_path="./3d")
fig2 = climb.plot_comprehensive_2d(save_path="./2d")
```

## Examples

See the `examples/` directory for detailed usage examples:

- `simple_example.py`: Basic usage with well-defined clusters
- `complex_mixed.py`: Handling mixed data with both convex and non-convex clusters
- `mixed_seeds.py`: Handling mixed data with both convex and non-convex clusters with seeds
- `compare_methods.py`: Comparing different exploratory clustering methods

## How It Works

CLiMB operates in two phases:

1. **Constrained Phase (KBound)**: A modified K-means that:
   - Uses seed points to guide initial clustering 
   - Applies density and distance constraints
   - Prevents centroids from drifting too far using radial thresholds
   - Supports customizable distance metrics through the distance_metric and metric_params parameters.
    - Handles advanced seed points via a dictionary structure for more controlled initialization.

2. **Exploratory Phase**: Uses density-based clustering methods to discover patterns in points not assigned during the first phase.

## Use Cases

CLiMB is particularly useful for:

- Datasets with partially known structure
- Astronomical data analysis
- Particle physics clustering
- Pattern discovery in scientific datasets
- Data exploration with prior knowledge

## Advanced Usage

### Using Different Exploratory Algorithms

```python
from CLiMB.core.CLiMB import CLiMB
from CLiMB.exploratory.HDBSCANExploratory import HDBSCANExploratory

# Create HDBSCAN exploratory algorithm
hdbscan = HDBSCANExploratory(min_cluster_size=5, min_samples=3)

# Use it with CLIMB
climb = CLiMB(
    constrained_clusters=3,
    exploratory_algorithm=hdbscan
)
```

### Parameter Tuning with Builder Pattern

```python
climb = CLiMB()
climb.set_density(0.3) \
     .set_distance(2.5) \
     .set_radial(1.0) \
     .set_convergence(0.1)
```

### Using Custom Distance Metrics
To use distance metrics other than Euclidean, you can use the distance_metric and metric_params parameters in the KBound class.

**Example with Mahalanobis Metric:**

```python
import numpy as np
from CLiMB.core.KBound import KBound

# ... (Load or generate your data X) ...

# Calculate the inverse covariance matrix (VI) for Mahalanobis
covariance_matrix = np.cov(X.T)
inv_covariance_matrix = np.linalg.inv(covariance_matrix)

# Initialize KBound with the Mahalanobis metric and parameters
kbound = KBound(
    n_clusters=3,
    distance_metric='mahalanobis',
    metric_params={'VI': inv_covariance_matrix}
)
kbound.fit(X)
```
If you do not provide metric_params for the Mahalanobis metric, the inverse covariance matrix will be automatically calculated on the input data X during fit.

**Example with Custom Metric:**
```python
import numpy as np
from CLiMB.core.KBound import KBound
from scipy.spatial.distance import euclidean

# Define your custom distance metric
def custom_distance(u, v):
    # Example: weighted Euclidean distance, where the first dimension counts double
    weight = np.array([2, 1, 1]) # Weights for each dimension
    return euclidean(u * weight, v * weight)

# Initialize KBound with the custom metric
kbound = KBound(
    n_clusters=3,
    distance_metric='custom',
    metric_params={'func': custom_distance}
)
kbound.fit(X)
```

### Advanced Usage of Seed Points with Dictionary
In addition to the list of seed points, you can now provide a dictionary to initialize centroids and associate specific seed points with each centroid. This offers more granular control over the initialization of constrained clustering.

**Example of Seed Point Dictionary:**
```python
import numpy as np
from CLiMB.core.KBound import KBound

# ... (Load or generate your data X and define initial centroids and seed points) ...

# Define a dictionary of seed points
# Keys are initial centroids (points), values are lists of seed points associated with that centroid
seed_dict = {
    tuple(initial_centroids[0]): [tuple(seed_point_cluster1_1), tuple(seed_point_cluster1_2)],
    tuple(initial_centroids[1]): [tuple(seed_point_cluster2_1)],
    tuple(initial_centroids[2]): [], # No specific seed points for this centroid
    # ... and so on for all centroids ...
}

# Initialize KBound with the seed point dictionary
kbound = KBound(
    n_clusters=len(initial_centroids),
    seeds=seed_dict
    # ... and so on for the other parameters
)
kbound.fit(X)
```

## Interpretability

CLiMB's decisions are not a black box, so the explanation is not a surrogate.
Phase 1 applies two explicit gates (density, distance) plus seed forcing; DBSCAN
applies a radius rule. Those are rules that can be replayed, so `explain()`
**reconstructs** them in closed form and checks the reconstruction against the
model. SHAP or LIME would fit a second model to imitate the first; here there is
nothing to imitate.

```python
explanation = climb.explain(X_scaled, feature_names=["f1", "f2", "f3"])
```

One row per point: `phase1_*` columns for every point, `phase2_*` for the points
Phase 1 rejected, and `final_label`. Call it *before* `inverse_transform()`,
which rewrites the stored centroids in place; pass `scaler=` to report the
feature columns in original units instead.

### Two numbering systems

Phase 2's algorithm numbers its clusters from 0, and CLiMB offsets them so they
do not collide with Phase 1's. Both appear in the table, and the column names
say which is which:

| column | numbering |
|---|---|
| `final_label`, `phase1_label`, `phase2_label` | CLiMB's — comparable with each other and with `get_labels()` |
| `phase2_algorithm_cluster` | the algorithm's own, starting at 0 |

Key any `{label: name}` map on CLiMB's numbering. The two overlap, so a map
built for the whole clustering will match the algorithm's cluster 0 *silently*
instead of raising, and label a Phase-2 discovery with a Phase-1 cluster's name.

### What is verified, and what is not

The package keeps three kinds of output apart rather than letting one borrow
another's credibility.

**Exact reconstructions** — replayed from the fitted model and verified on every
call, raising rather than returning a table that does not describe the model:

| | reconstructs | checked against |
|---|---|---|
| `KBound.decision_path` | gates and labels | `labels_` on non-seed points |
| `KBound.feature_attribution` | additive split of the metric | shares sum to the distance |
| `DBSCANExploratory.explain` | core/border/noise roles | `core_sample_indices_` |
| `OPTICSExploratory.explain` | core distances | `core_distances_` |

Seed points are excluded from the Phase-1 check because their label is copied,
not derived, which would make the check vacuous.

**Reported model state** — exact, but with nothing independent to check it
against. `HDBSCANExploratory.explain` reports membership probability, GLOSH
outlier score and cluster persistence; its labels come from the stability of a
condensed tree and there is no closed form to replay, so `fidelity_` stays
`None` rather than carrying a meaningless 1.0.

**Descriptive statistics** — `CLiMB.explain.descriptive` holds contrastive
signatures and effect sizes. These are computed after the fact, by our rules,
and nothing verifies them. A signature says *these clusters differ on this
feature*; it does not say *this is why the point was assigned*. For that, use
`feature_attribution`, which decomposes the metric the model actually used.

### Why the three algorithms explain differently

`explain()` is abstract on `ExploratoryClusteringBase` and implemented
separately, not generalised. Phase 2's algorithms do not share a decision
structure: DBSCAN assigns roles at one fixed radius, OPTICS refuses to fix a
radius and orders points by reachability, HDBSCAN decides membership by cluster
stability across every scale at once. Common columns would mean inventing a
core/border split for two algorithms that have none.

### Plots

The helpers in `CLiMB.explain` are domain-neutral: axis and colourbar text is
always the caller's, never inferred from the features.

```python
from CLiMB.explain import plot_gate_accounting, plot_margin_map

plot_gate_accounting(explanation, column="phase1_gate")
plot_margin_map(explanation, x="f1", y="f2", value="phase1_distance_margin",
                xlabel="f1 (your units)", ylabel="f2 (your units)")
```
## Documentation
Full documentation is available at [https://climb-astro.readthedocs.io](https://climb-astro.readthedocs.io).

## License

MIT

## Project status
![Test Status](https://github.com/LorenzoMonti/CLiMB/actions/workflows/test.yml/badge.svg)
[![Documentation Status](https://readthedocs.org/projects/climb-astro/badge/?version=latest)](https://climb-astro.readthedocs.io/en/latest/?badge=latest)
[![PyPI version](https://badge.fury.io/py/climb-astro.svg)](https://badge.fury.io/py/climb-astro)

## Citations
```
@article{monti2026climb,
  title={CLiMB: A domain-informed novelty detection clustering framework for galactic archaeology and scientific discovery},
  author={Monti, Lorenzo and Muraveva, Tatiana and Sheridan, Brian and Massari, Davide and Garofalo, Alessia and Clementini, Gisella and Michelucci, Umberto},
  journal={Astronomy and Computing},
  pages={101118},
  year={2026},
  publisher={Elsevier}
}

@article{muraveva2026probing,
  title={Probing the Milky Way halo with RR Lyrae stars from Gaia Data Release 3},
  author={Muraveva, T and Monti, L and Massari, D and De Leo, M and Garofalo, A and Clementini, G and Ceccarelli, E and Michelucci, U},
  journal={Astronomy \& Astrophysics},
  volume={708},
  pages={A384},
  year={2026},
  publisher={EDP Sciences}
}
```
