Metadata-Version: 2.4
Name: bigclust2
Version: 0.1.1
Summary: A graphical interface for interactively exploring clusterings of high-dimensional connectomic data
Author-email: Philipp Schlegel <pms70@cam.ac.uk>
License-Expression: GPL-3.0-or-later
Project-URL: Homepage, https://github.com/flyconnectome/bigclust2
Project-URL: Repository, https://github.com/flyconnectome/bigclust2
Project-URL: Issues, https://github.com/flyconnectome/bigclust2/issues
Keywords: connectomics,clustering,neuroscience,visualization,gui,embeddings
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: X11 Applications :: Qt
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Visualization
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pandas>=2.3.3
Requires-Dist: pyside6>=6.8.2.1
Requires-Dist: requests>=2.32.5
Requires-Dist: polars>=1.36.1
Requires-Dist: cmap>=0.6.2
Requires-Dist: pygfx>=0.14.0
Requires-Dist: llvmlite>=0.46.0
Requires-Dist: numba>=0.63.1
Requires-Dist: umap-learn>=0.5.9.post2
Requires-Dist: scipy>=1.15.3
Requires-Dist: gevent==25.5.1
Requires-Dist: cloud-volume>=12.8.0
Requires-Dist: octarine3d>=0.5.0
Requires-Dist: octarine-navis-plugin>=0.1.5
Requires-Dist: navis>=1.10.0
Requires-Dist: dvidtools>=0.6.0
Requires-Dist: pyperclip>=1.11.0
Requires-Dist: nglscenes>=0.5.1
Requires-Dist: pyarrow>=23.0.0
Requires-Dist: pyqtgraph>=0.14.0
Requires-Dist: scikit-learn>=1.7.2
Requires-Dist: networkx>=3.4.2
Requires-Dist: sea-serpent>=0.4.4
Requires-Dist: plotly>=6.7.0
Requires-Dist: neuprint-python>=0.6.2
Provides-Extra: clio
Requires-Dist: clio-py>=0.1.0; extra == "clio"
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Requires-Dist: pytest-cov; extra == "dev"
Requires-Dist: ruff; extra == "dev"
Dynamic: license-file

# BigClust 2.0

> [!NOTE]
> This project is under active development but that shouldn't stop you from using it to explore your clustering datasets. If you have any feedback or questions, please open an issue here on GitHub.

`bigclust2` is a re-design of [`bigclust`](https://github.com/flyconnectome/bigclust), a graphical interface for interactively exploring clusterings of high-dimensional connectomic data.
Typically this means morphological or connectivity-based embeddings but it can be used for any kind of distances or features.

Highlights:
- **Interactive 2D scatter plots**: Explore large clusterings interactively with zoom, pan, and selection.
- **Neuroglancer-like 3D viewer**: Visualize neuron morphology in a 3D viewer.
- **Widgets for days**: All the tools you need to explore and cluster your data: cluster methods, dimensionality reduction, fidelity metrics, feature selection, connectivity explorer, etc.
- **Annotations**: Push annotation straight to supported backends (e.g. Clio or SeaTable) or export them as CSV/Parquet files.

![BigClust 2.0 GUI](./_static/screenshot.png)

## Version 2.0 Notes
A totally reworked GUI aside, this new version also fundamentally changes how data is represented:
previously, data artifacts (distances, features, etc.) had to be manually loaded and passed to BigClust widgets.
For this new version, we have switched to a Neuroglancer-like approach where data sources are whole directories
(local or remote) containing both the data itself as well as metadata files describing the setup. Here is a simple
example structure:

```
/my_clustering/
    info                <- JSON-formatted settings for the dataset
    meta.parquet        <- per-neuron metadata in Parquet (recommended) or Apache Arrow Feather format
    distances.parquet   <- pairwise distances in Parquet (recommended) or Apache Arrow Feather format (optional)
    embeddings.parquet  <- low-dimensional embeddings in Parquet (recommended) or Apache Arrow Feather format (optional)
    features.parquet    <- high-dimensional features in Parquet (recommended) or Apache Arrow Feather format (optional)
```

See the [Data Format](./DATA_FORMAT.md) documentation for details on the expected files and how to set up your project.

## Usage

First make sure you have the Python package manager `uv` [installed](https://docs.astral.sh/uv/getting-started/installation/). Then run:

```bash
uvx bigclust2@latest
```

To work with the latest development version of `bigclust2`, you can run it directly from this repository:


```bash
uvx --from git+https://github.com/flyconnectome/bigclust2@main bigclust2
```

> [!TIP]
> Note the `@main` in above command? This is asking `uvx` to always use the latest version of `bigclust2`.
> Because of how `uvx` works, this can lead to slow start-up times when there are new releases. You can
> avoid this by pinning to a specific version (e.g. `@0.1.0`) or a specific commit (e.g. `@02ea911`).

### Controls

Most UI elements are hopefully self-explanatory - when in doubt look for the tooltip.

#### Scatterplot

- left click to move the view
- shift + left click to draw a selection box (add + cmd to add to selection)
- shift + control + left click to draw a lasso selection (add + cmd to add to selection)
- `ESC` to deselect all points

- `C` to toggle the control panel
- `L` to toggle labels
- left/right arrows increase/decrease font size of labels
- up/down arrows increase/decrease marker size
- double-click on a label to highlight points with the same label
- shift + double-click on a label to select points with the same label
- CMD/control + shift + double-click on a label to add points with the same label to the current selection

#### 3D Viewer

- left click + hold to rotate the view
- middle button + hold to pan
- scroll to zoom in and out
- `C` to toggle the legend
- to align the view: `1` (front), `2` (side), `3` (top)

## Troubleshooting

| Error  | Solution |
| ------ | -------  |
| Running `uvx ...` fails with an error containing `realpath: command not found` | If you're on Mac, make sure your OS version is at least 13.x |

## Development

1. Clone this repository
2. `cd bigclust2` to change into this directory
3. `uv run run.py --debug` to start the GUI

## Data Format

Data sources are whole directories (local or remote) containing the data itself plus metadata files describing the setup. The full specification — the `info` file, the `meta`/`distances`/`embeddings`/`features` files, KNN graphs, multiple embeddings, multiple datasets, and best practices for Parquet formatting — lives in [DATA_FORMAT.md](./DATA_FORMAT.md).

## Ideas / TODOs
- [x] Enable multiple embeddings per dataset (with a dropdown to select embedding)
- [x] Allow selecting sets of features (e.g. upstream vs downstream or isomorphic vs dimorphic connections; this could simply use the multi-columns)
- [x] Support pushing updated annotations back to annotation backends (e.g. Clio/FlyTable)
- [ ] Add support for Graph Exploration
- [ ] Allow users to change project settings (e.g. neuroglancer sources) before or after loading
- [ ] Support loading up-to-date annotations when opening a project (e.g. from Clio or Neuprint)
- [x] Add more detailed documentation about data formats and structure
- [ ] Caching system for remote data sources for faster start-up times
  - this turns out to not really be a problem from the client side; could still implement to reduce traffic on the server side
- [ ] Support sharing figure state (e.g. `uvx bigclust --state <state_id>`)
- [X] Fine-control over hover info
