Metadata-Version: 2.4
Name: neurovlm
Version: 0.0.5
Summary: Neuroimaging vision-language models.
Author-email: Ryan Hammonds <ryanphammonds@gmail.com>
License-Expression: Apache-2.0
License-File: LICENSE
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.13
Requires-Python: <3.14,>=3.10
Requires-Dist: accelerate>=1.12.0
Requires-Dist: adapters
Requires-Dist: datasets>=4.8.4
Requires-Dist: huggingface-hub
Requires-Dist: matplotlib
Requires-Dist: nibabel
Requires-Dist: nilearn
Requires-Dist: numpy
Requires-Dist: pandas
Requires-Dist: pyarrow
Requires-Dist: safetensors
Requires-Dist: torch
Requires-Dist: transformers
Provides-Extra: cogatlas
Requires-Dist: networkx; extra == 'cogatlas'
Provides-Extra: coords
Requires-Dist: nimare; extra == 'coords'
Provides-Extra: crawling
Requires-Dist: beautifulsoup4; extra == 'crawling'
Requires-Dist: requests; extra == 'crawling'
Requires-Dist: tqdm; extra == 'crawling'
Provides-Extra: docs
Requires-Dist: myst-nb<2,>=1.1; extra == 'docs'
Requires-Dist: myst-parser<4,>=2.0; extra == 'docs'
Requires-Dist: sphinx-book-theme<2,>=1.1; extra == 'docs'
Requires-Dist: sphinx-copybutton<1,>=0.5; extra == 'docs'
Requires-Dist: sphinx-design<1,>=0.6; extra == 'docs'
Requires-Dist: sphinx<9,>=7.2; extra == 'docs'
Provides-Extra: full
Requires-Dist: accelerate; extra == 'full'
Requires-Dist: beautifulsoup4; extra == 'full'
Requires-Dist: ipython; extra == 'full'
Requires-Dist: matplotlib; extra == 'full'
Requires-Dist: networkx; extra == 'full'
Requires-Dist: nimare; extra == 'full'
Requires-Dist: nltk; extra == 'full'
Requires-Dist: ollama; extra == 'full'
Requires-Dist: requests; extra == 'full'
Requires-Dist: rouge-score; extra == 'full'
Requires-Dist: scikit-image; extra == 'full'
Requires-Dist: scikit-learn; extra == 'full'
Requires-Dist: seaborn; extra == 'full'
Requires-Dist: tqdm; extra == 'full'
Provides-Extra: llm
Requires-Dist: accelerate; extra == 'llm'
Requires-Dist: ollama; extra == 'llm'
Provides-Extra: metrics
Requires-Dist: nltk; extra == 'metrics'
Requires-Dist: rouge-score; extra == 'metrics'
Requires-Dist: scikit-image; extra == 'metrics'
Requires-Dist: scikit-learn; extra == 'metrics'
Provides-Extra: notebook
Requires-Dist: ipython; extra == 'notebook'
Requires-Dist: jupyter; extra == 'notebook'
Requires-Dist: tqdm; extra == 'notebook'
Provides-Extra: viz
Requires-Dist: matplotlib; extra == 'viz'
Requires-Dist: seaborn; extra == 'viz'
Description-Content-Type: text/markdown

# NeuroVLM

NeuroVLM maps between neuroimaging activation maps and neuroscience text.

![model](https://github.com/neurovlm/neurovlm_data/blob/13dd7769f9603c036a9338b7da4adc2f3a03ec94/docs/model.png)


## System Requirements

- python >=3.10, <3.14
- ubuntu-latest, macos-latest
- GPU (NVIDIA or Apple MPS) or CPU


## Installation

Minimal, inference-only installation:

```bash
pip install neurovlm
```

With optional dependencies needed to train and reproduce analyses:

```bash
pip install "neurovlm[full]"
```

Installation take a couple minutes. After installation, calling `neurovlm.data.fetch_data()` will fetch datasets and models from huggingface, which will be slower.


## Demo

See [here](https://github.com/neurovlm/neurovlm/blob/main/docs/tutorials/00_quickstart.ipynb) for the introductory notebook that walks through using all NeuroVLM models. In short:

Fetch NeuroVLM's datasets and models:

```python
from neurovlm.data import fetch_data
fetch_data()
```

Use the four text/brain inference paths:

```python
from neurovlm import NeuroVLM
from neurovlm.data import load_latent

nvlm = NeuroVLM(device="cuda") # use device="cpu" if GPU not available

# Text-to-brain generation (MSE)
brain_map = nvlm.text("auditory processing").to_brain(head="mse")
brain_map.plot(0, threshold=0.1)

# Brain-to-text generation (QFormer)
auditory = load_latent("networks_neuro")["Du"]["AUD"]
description = nvlm.brain(auditory).to_text(head="qformer")
print(description)

# Text-to-brain retrieval (InfoNCE)
brain_matches = nvlm.text("auditory processing").to_brain(head="infonce")
df_text_to_brain = brain_matches.top_k(3)

# Brain-to-text retrieval (InfoNCE)
text_matches = nvlm.brain(auditory).to_text(head="infonce")
df_brain_to_text = text_matches.top_k(3)
```

## Documentation

See the [docs](https://neurovlm.github.io/neurovlm/) for the [API](https://neurovlm.github.io/neurovlm/api.html) and [tutorials](https://neurovlm.github.io/neurovlm/tutorials/index.html).


## Reproducibility

All analyses are in Juptyer notebooks. Their are three directories:

1. `docs/01_data`: Fetch raw data and preprocess
2. `docs/02_models`: Trains all models
3. `docs/03_evaluation`: Evaluates models and reproduces publication figures.

## License

Apache-2.0 (`LICENSE`).
