Metadata-Version: 2.4
Name: beast-backbones
Version: 1.4.0
Summary: Behavioral analysis via self-supervised pretraining of transformers
License-Expression: MIT
License-File: LICENSE
Keywords: machine learning,deep learning,computer_vision
Author: Yanchen Wang
Requires-Python: >=3.10
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Scientific/Engineering :: Image Processing
Provides-Extra: dev
Requires-Dist: imaug
Requires-Dist: jaxtyping
Requires-Dist: lightning (>=2.5,<3.0)
Requires-Dist: numpy (>=2.0.0)
Requires-Dist: opencv-python-headless
Requires-Dist: pillow
Requires-Dist: pre-commit ; extra == "dev"
Requires-Dist: pydantic
Requires-Dist: pyright ; extra == "dev"
Requires-Dist: pytest ; extra == "dev"
Requires-Dist: pytest-cov ; extra == "dev"
Requires-Dist: pytest-html ; extra == "dev"
Requires-Dist: requests ; extra == "dev"
Requires-Dist: ruff ; extra == "dev"
Requires-Dist: scikit-learn
Requires-Dist: tensorboard
Requires-Dist: toml
Requires-Dist: torchvision
Requires-Dist: tqdm
Requires-Dist: transformers
Requires-Dist: urllib3 (>=2.7.0) ; extra == "dev"
Project-URL: Documentation, https://github.com/paninski-lab/beast
Project-URL: Repository, https://github.com/paninski-lab/beast
Description-Content-Type: text/markdown

# beast

![GitHub](https://img.shields.io/github/license/paninski-lab/beast)
[![codecov](https://codecov.io/gh/paninski-lab/beast/branch/main/graph/badge.svg)](https://codecov.io/gh/paninski-lab/beast)
![PyPI](https://img.shields.io/pypi/v/beast-backbones)

**Be**havioral **a**nalysis via **s**elf-supervised pretraining of **t**ransformers

`beast` is a package for pretraining vision transformers on unlabeled data to provide backbones 
for downstream tasks like pose estimation, action segmentation, and neural encoding.

See the ICLR paper [here](https://openreview.net/pdf?id=AeqPIRKUni). 

## Installation

### Step 1: Install ffmpeg
First, check to see if you have ffmpeg installed by typing the following in the terminal:

```commandline
ffmpeg -version
```

If not, install:

```commandline
sudo apt install ffmpeg
```

### Step 2: Create a conda environment

First, [install anaconda](https://docs.anaconda.com/free/anaconda/install/index.html).

Next, create and activate a conda environment:

```commandline
conda create --yes --name beast python=3.10
conda activate beast
```

### Step 3: Download and install
Move to your home directory (or wherever you would like to download the code) and install via Github clone or through PyPI.

For Github cloning:

```commandline
git clone https://github.com/paninski-lab/beast
cd beast
pip install -e .
```

For installation through PyPI:

```commandline
pip install beast-backbones
```

## Usage

`beast` comes with a simple command line interface. To get more information, run
```commandline
beast -h
```

### Extract frames

Extract frames from a directory of videos to train `beast` with.

```commandline
beast extract --input <video_dir> --output <output_dir> [options]
```

Type "beast extract -h" in the terminal for details on the options.

### Train a model

You will need to specify a config path; see the `configs` directory for examples.

```commandline
beast train --config <config_path> [options]
```

Type "beast train -h" in the terminal for details on the options.

### Run inference

Inference on a single video or a directory of videos: 

```commandline
beast predict --model <model_dir> --input <video_path> [options]
```

Inference on (possibly nested) directories of images: 

```commandline
beast predict --model <model_dir> --input <video_path> [options]
```

Type "beast predict -h" in the terminal for details on the options.

---

## Contributing

See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines on setting up a development environment,
code style, and submitting pull requests.

## Funding

We are grateful for support from the following:
* Gatsby Charitable Foundation GAT3708
* [NIH R50NS145433](https://reporter.nih.gov/search/Hmj4KMmLv0evcYPlPEDa-Q/project-details/11240675)
* [NIH U19NS123716](https://reporter.nih.gov/search/Hmj4KMmLv0evcYPlPEDa-Q/project-details/11141703)
* [NSF 1707398](https://ui.adsabs.harvard.edu/abs/2017nsf....1707398A/abstract)
* [The NSF AI Institute for Artificial and Natural Intelligence](https://ui.adsabs.harvard.edu/abs/2023nsf....2229929Z/abstract)
* Simons Foundation
* Wellcome Trust 216324
* Zuckerman Institute (Columbia University) Team Science

