Metadata-Version: 2.4
Name: napari-deeplabcut
Version: 0.3.1.0
Summary: napari + DeepLabCut annotation tool
Author-email: "Team DeepLabCut, led by Jessy Lauer" <admin@deeplabcut.org>
License-Expression: LGPL-3.0-only
Project-URL: Bug Tracker, https://github.com/DeepLabCut/napari-deeplabcut/issues
Project-URL: Documentation, https://github.com/DeepLabCut/napari-deeplabcut#README.md
Project-URL: Source Code, https://github.com/DeepLabCut/napari-deeplabcut
Project-URL: User Support, https://github.com/DeepLabCut/napari-deeplabcut/issues
Classifier: Development Status :: 4 - Beta
Classifier: Framework :: napari
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Scientific/Engineering :: Image Processing
Classifier: Topic :: Scientific/Engineering :: Visualization
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: dask-image
Requires-Dist: matplotlib>=3.3
Requires-Dist: napari==0.6.6
Requires-Dist: natsort
Requires-Dist: numpy<2,>=1.18.5
Requires-Dist: opencv-python-headless
Requires-Dist: pandas
Requires-Dist: pyside6>=6.4.2
Requires-Dist: pyyaml
Requires-Dist: qtpy>=2.4
Requires-Dist: scikit-image
Requires-Dist: scipy
Requires-Dist: shiboken6>=6.4.2
Requires-Dist: tables
Provides-Extra: dev
Requires-Dist: pillow; extra == "dev"
Requires-Dist: pre-commit; extra == "dev"
Requires-Dist: pytest; extra == "dev"
Requires-Dist: pytest-cov; extra == "dev"
Requires-Dist: pytest-qt; extra == "dev"
Requires-Dist: tox; extra == "dev"
Provides-Extra: tracking
Requires-Dist: torch; extra == "tracking"
Dynamic: license-file

# napari-deeplabcut - Keypoint annotation tool for pose estimation

<img src="https://images.squarespace-cdn.com/content/v1/57f6d51c9f74566f55ecf271/1d409ffe-c9f4-47e1-bde2-3010c1c40455/naparidlc.png?format=750w" width="450" title="napari-deeplabcut" alt="napari+deeplabcut" align="right" vspace="80">

[📚 Plugin Documentation](https://deeplabcut.github.io/DeepLabCut/docs/gui/napari_GUI.html) |
[🛠️ DeepLabCut Installation](https://deeplabcut.github.io/DeepLabCut/docs/installation.html) |
[🌎 DeepLabCut Home Page](https://www.deeplabcut.org) |

[![License: LGPL-3.0](https://img.shields.io/badge/License-LGPL%203.0-blue.svg)](https://www.gnu.org/licenses/lgpl-3.0)
[![PyPI](https://img.shields.io/pypi/v/napari-deeplabcut.svg?color=green)](https://pypi.org/project/napari-deeplabcut)
[![Python Version](https://img.shields.io/pypi/pyversions/napari-deeplabcut.svg?color=green)](https://python.org)
[![tests](https://github.com/DeepLabCut/napari-deeplabcut/actions/workflows/test_and_deploy.yml/badge.svg?branch=main)](https://github.com/DeepLabCut/napari-deeplabcut/actions/workflows/test_and_deploy.yml)
[![codecov](https://codecov.io/gh/DeepLabCut/napari-deeplabcut/branch/main/graph/badge.svg)](https://codecov.io/gh/DeepLabCut/napari-deeplabcut)
[![napari hub](https://img.shields.io/endpoint?url=https://api.napari-hub.org/shields/napari-deeplabcut)](https://napari-hub.org/plugins/napari-deeplabcut)

A napari plugin for keypoint annotation and label refinement, also used within DeepLabCut.

---

## Installation

If you installed `DeepLabCut[gui]`, this plugin is already included.

You can also install `napari-deeplabcut` as a standalone keypoint annotation plugin without using the full DeepLabCut GUI.

### Standard install

Using `pip` (e.g. in a `conda` environment):

```bash
pip install napari-deeplabcut
```

Using `uv`:

```bash
uv venv -p 3.12 # create a new virtual environment with Python 3.12
source .venv/bin/activate # activate the virtual environment. Use the relevant command for your OS/shell if different.
uv pip install napari-deeplabcut
```

> [!NOTE]
> A conda environment or uv venv is not strictly required. Please use your preferred package manager!

### Latest development version

Using `pip`:

```bash
pip install git+https://github.com/DeepLabCut/napari-deeplabcut.git
```

---

## Usage

### Main documentation

Please see the [official plugin documentation](https://deeplabcut.github.io/DeepLabCut/docs/gui/napari/basic_usage.html) for detailed usage instructions.

### Standalone usage

Start napari:

```bash
napari
```

Then activate the plugin in:

> **Plugins → napari-deeplabcut: Keypoint controls**

Accepted files such as `config.yaml`, image folders, videos, and `.h5` annotation files can be loaded either by dragging them onto the canvas or through the **File** menu.

> [!TIP]
> The widget opens automatically when drag-and-dropping a compatible labeled-data folder.

### Recommended way to get started

The easiest way to start labeling from scratch is:

1. Open (or drag-and-drop) an image-only folder from your computer, or within a DeepLabCut project's `labeled-data` directory
  - This means that only the images are loaded, without any existing annotations
2. Open (or drag-and-drop) the `config.yaml` from your project

This creates:
- an **Image** layer with the images (or video frames), and
- an empty **Points** layer populated with the keypoint metadata from the config.

You may then start annotating in the points layer that was created.

> [!NOTE]
> If you load a folder from outside a DeepLabCut project and try to save a Points layer, you will be prompted to provide the config.yaml file
> used by the project. You may then move the labeled data folder into your project directory for downstream use.

[🎥 DEMO](https://youtu.be/hsA9IB5r73E)

---

## Tools and shortcuts

- `2` / `3`: switch between labeling and selection mode when a Points layer is active
- `4`: enable pan & zoom
- `M`: cycle through sequential, quick, and cycle annotation modes
- `E`: toggle edge coloring
- `F`: toggle between individual and body-part coloring modes
- `V`: toggle visibility of the selected layer
- `Backspace`: delete selected point(s)
- `Ctrl+C` / `Ctrl+V`: copy and paste selected points
- Double-click the current frame number to jump to a specific frame

> [!TIP]
> Press the "View shortcuts" button in the dock widget for a reference.

Additional dock controls include:

- **Warn on overwrite**: enable or disable confirmation prompts when saving would overwrite existing annotations
- **Show trails**: display keypoint trails over time in the main viewer
- **Show trajectories**: open a trajectory plot in a separate dock widget
- **Show color scheme**: display the active/configured color mapping reference
- **Video tools**: extract the current frame and store crop coordinates for videos

---

## Saving layers

Use:

> **File → Save Selected Layer(s)...**

or the shortcut:

```text
Ctrl+S
```

### Keypoint save behavior

Keypoint annotations are automatically saved into the corresponding dataset folder as:

```text
CollectedData_<ScorerName>.h5
```

For convenience, the companion `.csv` file is written in the same folder.

### Important notes

- DeepLabCut uses the **H5** file as the authoritative annotation file.
- Before saving, make sure the **Points** layer you want to save is selected.
  - The plugin will not save if several Points layers are selected at the same time, to avoid ambiguity.
- Saving a `machinelabels...` layer does **not** write back to the machine labels file.
  Instead, refined annotations are written into the appropriate `CollectedData...` file.
- If saving would overwrite existing annotations, the plugin will prompt for confirmation.
  - While labeling, confirmation can be disabled by unchecking the "Warn on overwrite" option in the dock widget.
- Several plugin functions implicitly expect `config.yaml` to be present two folders up from the saved `CollectedData...` file, so make sure to keep the config in the project directory structure for best results. Fallback behaviors are present but may not cover all edge cases.
  - If you save a Points layer without a config file present in the expected location, you will be prompted to provide the path to the config file that matches the dataset you are working on. The plugin will then save the points and metadata into the correct folder based on the config path provided. Afterwards, it is recommended to move the dataset folder into the correct location within the project directory structure for best compatibility with other DeepLabCut functions. Please edit the `config.yaml` file if needed to update the paths to the videos and image folders.

---

## Video support

Videos can be opened directly in the GUI.

When a video is loaded, the plugin enables a small video action panel that can be used to:

- Extract the current frame into the dataset
- Optionally export existing machine labels for that frame
- Define and save crop coordinates to the DLC `config.yaml`

Keypoints in video-based workflows can be edited and saved in the same way as ordinary image-folder workflows.

---

## Workflow (outside of DLC GUI)

Suggested workflows depend on what is already present in the dataset folder.

Please note this describes the workflow when napari is launched as a standalone application, outside of the DeepLabCut GUI.

### 1) Labeling from scratch

Use this when the image folder does **not** yet contain a `CollectedData_<ScorerName>.h5` file.

1. Open a folder of extracted images
2. Open the corresponding DeepLabCut `config.yaml`
3. Select the created **Points** layer
4. Start labeling
5. Save the points layer with `Ctrl+S`

After saving, the folder should now contain:

```text
CollectedData_<ScorerName>.h5
CollectedData_<ScorerName>.csv
```

---

### 2) Resuming labeling

Use this when the folder already contains a `CollectedData_<ScorerName>.h5` file.

Open (or drag-and-drop) the folder in napari. The existing keypoint metadata and annotations will be loaded from the H5 file, so loading `config.yaml` is not needed nor recommended.

However, loading the config is still useful if:

- The project’s bodyparts changed
- You would like to refresh the configured color scheme from the project config

---

### 3) Refining machine labels

Use this when the folder contains a machine predictions file such as:

```text
machinelabels-iter<...>.h5
```

Open the folder in napari.

If both a `CollectedData...` file and a `machinelabels...` file are present:

- Edit the `machinelabels` layer to refine predictions
- Optionally use edge coloring (`E`) to highlight low-confidence labels
- Save the selected `machinelabels` layer to merge refinements into `CollectedData`

If the folder contains only `machinelabels...` and no `CollectedData...`, refined annotations will still be saved into a new `CollectedData...` target.

---

## Workflow flowchart

```mermaid
%%{init: {"flowchart": {"htmlLabels": false}} }%%
graph TD
  id1[What stage of labeling?]
  id2[deeplabcut.label_frames]
  id3[deeplabcut.refine_labels]
  id4[Add labels to, or modify in,
 `CollectedData...` layer and save that layer]
  id5[Modify labels in `machinelabels` layer and save
 which will create or update `CollectedData...`]
  id6[Have you refined some labels from the most recent iteration and saved already?]
  id7["All extracted frames are already saved in `CollectedData...`.
1. Hide or remove all `machinelabels` layers.
2. Continue working in `CollectedData`."]
  id8["1. Keep only the most recent `machinelabels` layer.
2. Select it and press `E` to show edges.
3. Refine labels in `machinelabels`.
4. Save to merge into `CollectedData`.
- If you revisit the dataset later, you can continue working in `CollectedData`."]

  id1 -->|I need to manually label new frames
 or fix existing labels|id2
  id1 -->|I need to refine outlier frames
 from analyzed videos|id3
  id2 --> id4
  id3 -->|I only have a `machinelabels...` file|id5
  id3 -->|I have both `machinelabels` and `CollectedData` files|id6
  id6 -->|yes|id7
  id6 -->|no, I just extracted outliers|id8
```

---

## Labeling multiple image folders

Only one dataset folder should be worked on at a time.

After finishing a folder:

1. Save the relevant **Points** layer
2. Remove the current layers from the viewer
3. Open the next folder

This keeps plugin operation and saving unambiguous.

---

## Defining crop coordinates

To store crop coordinates in a DLC project:

1. Open the video from the project’s `videos` folder
2. Enable cropping in the video tools
3. Draw a rectangle in the newly created crop layer (the tool is selected by default)
4. Click **Store crop coordinates** after checking the coordinates in the widget.

The crop coordinates are then written back to the project configuration.

---

## Contributing

Contributions are welcome.

Tests can be run locally with [tox].
Please note we use pre-commit hooks to run linters and formatters on changed files, so make sure to install the pre-commit dependencies:

```bash
pip install pre-commit
pre-commit install
```

### Development install

Clone the repository and install it in editable mode.

Using `pip`:

```bash
pip install -e .
```

If you need development dependencies as well, use the project’s `dev` extra:

```bash
pip install -e .[dev] # includes pre-commit
```

## License

Distributed under the terms of the [LGPL-3.0](https://www.gnu.org/licenses/lgpl-3.0).

## Issues

If you encounter any problems, please [file an issue](https://github.com/DeepLabCut/napari-deeplabcut/issues) with a detailed description and, if possible, a minimal reproducible example.

## Acknowledgements

This [napari](https://github.com/napari/napari) plugin was originally generated with [Cookiecutter](https://github.com/audreyr/cookiecutter) using [@napari](https://github.com/napari)'s [cookiecutter-napari-plugin](https://github.com/napari/cookiecutter-napari-plugin) template.

We thank the Chan Zuckerberg Initiative (CZI) for funding the initial development of this work!

[tox]: https://tox.readthedocs.io/en/latest/
