Metadata-Version: 2.4
Name: deep-learning-toolkit
Version: 0.5.0
Summary: Deep Learning Toolkit: Reusable PyTorch building blocks for artificial intelligence and scientific machine learning: networks, losses, training loops, and utilities.
Keywords: artificial intelligence,deep learning,machine learning,neural networks,scientific computing,pytorch
Author: Johann Rudi
Author-email: Johann Rudi <jrudi@vt.edu>
License-Expression: Apache-2.0
License-File: LICENSE
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Requires-Dist: matplotlib>=3,<4
Requires-Dist: prettytable>=3,<4
Requires-Dist: pyyaml>=6,<7
Requires-Dist: torch>=2,<3
Requires-Dist: tqdm>=4,<5
Requires-Dist: flow-matching>=1.0.10,<2 ; extra == 'diffusion'
Requires-Dist: torch-kde>=0.1.5 ; extra == 'kde'
Requires-Python: >=3.11
Project-URL: Homepage, https://github.com/johannrudi/deep-learning-toolkit
Project-URL: Repository, https://github.com/johannrudi/deep-learning-toolkit
Project-URL: Issues, https://github.com/johannrudi/deep-learning-toolkit/issues
Project-URL: Changelog, https://github.com/johannrudi/deep-learning-toolkit/blob/main/CHANGELOG.md
Provides-Extra: diffusion
Provides-Extra: kde
Description-Content-Type: text/markdown

# Deep Learning Toolkit

<!-- NOTE: github badges work with private repos -->
<!--
[![CI](https://github.com/johannrudi/deep-learning-toolkit/actions/workflows/ci.yml/badge.svg)](https://github.com/johannrudi/deep-learning-toolkit/actions/workflows/ci.yml)
-->
<!-- NOTE: shields.io badges work *only* with public repos -->
[![CI](https://img.shields.io/github/actions/workflow/status/johannrudi/deep-learning-toolkit/ci.yml?style=for-the-badge&label=CI)](https://github.com/johannrudi/deep-learning-toolkit/actions/workflows/ci.yml)

Reusable [PyTorch](https://pytorch.org/) building blocks for artificial intelligence & scientific machine learning: networks, losses, training loops, and utilities.

The *Deep Learning Toolkit* is a Python library of reusable [PyTorch](https://pytorch.org/) components for artificial intelligence and scientific machine learning. It is designed to be composed into research codes (not a standalone application) and accelerate development of such codes.

The package provides:

- **Network architectures**: multilayer perceptrons with residual and attention blocks, 1D and 2D convolutional networks, UNets, 1D transformers with patch embeddings, autoencoders, and more.
- **Training and optimization**: epoch- and batch-level training loops with checkpointing and validation hooks, distributed training, GAN training loops, and multi-stage learning rate schedulers.
- **Supporting components**: loss functions, evaluation metrics, plotting helpers, and configuration management.

Network modules share a consistent activation-aware parameter initialization scheme, and training routines return structured logs of per-epoch and per-batch loss statistics.

---

## Installing the `deep-learning-toolkit`

### Requirements

- Python version `>=3.11`

### Runtime dependencies

- `matplotlib` version `>=3,<4`
- `prettytable` version `>=3,<4`
- `pyyaml` version `>=6,<7`
- **`torch` version `>=2,<3`**
- `tqdm` version `>=4,<5`

### Install commands using `pip`

```sh
pip install deep-learning-toolkit
```

#### Install with optional extras

Dependencies for generative diffusion models:

```sh
pip install deep-learning-toolkit[diffusion]
```

Dependencies for kernel density estimation:

```sh
pip install deep-learning-toolkit[kde]
```

---

## Importing and using `dlk`

To use the toolkit, import its modules in your Python code like this:

```py
from dlk.nets.mlp import MLPNet
from dlk.opt.train import train_epochs

# load your data
...

# create the model
net = MLPNet(input_size=784, output_size=10)

# train the model
train_epochs(n_epochs=100, net=net, dataloader=..., optimizer=..., loss_fn=...)

# evaluate on your data
...
```

---

## Architecture

### Neural network architectures &rarr; `dlk/nets/`

- `mlp.py`: Multilayer Perceptron (MLPNet, MLPNet_MultIn, MLPResNet with residual and attention blocks)
- `autoencoder.py`: Generic autoencoder wrapper for encoder/decoder pairs
- `conv1d.py`, `conv2d.py`: 1D/2D convolutional networks and UNet components (Downsample, Upsample)
- `unet.py`: Complete UNet implementations (older UNet1D/UNet2D and newer UNetXd_2025 architecture)
- `transformer1d.py`: 1D transformer networks with patch embeddings and multi-head attention
- `efficientnet.py`: EfficientNet architecture

#### Network initialization

All network modules follow a consistent pattern:

- Constructor calls `self.init_parameters()` at the end
- `init_parameters()` uses Xavier initialization with gain calculated from activation functions
- Utility functions `_get_gain()` and `_set_init_parameters()` handle activation-aware initialization

### Training and optimization &rarr; `dlk/opt/`

- `train.py`: Training loops (`train_epochs`, `train_batches`) with checkpointing and validation hooks
- `train_gan.py`: GAN-specific training loops
- `scheduler.py`: Learning rate schedulers (multi-stage: linear warmup, constant, cosine annealing)

#### Logging of the training progress

Training functions return detailed logging dictionaries (`dlog`) containing:

- Per-epoch loss statistics (`loss_mean`, `loss_std`)
- Batch-level logs nested in `batch_dlog`
- Total training time in `time_train`
- Checkpointing saves model and optimizer states at specified intervals

### Additional components of the package

- `dlk/mgmt/`: Management of configuration parameter loading/saving, logging, etc.
- `dlk/loss/`: Loss functions
- `dlk/metrics/`: Metrics for evaluating trained nets

---

## Development

### Set up a development environment

Obtain a clone of the [git repository](https://github.com/johannrudi/deep-learning-toolkit/). This project is managed with [uv](https://docs.astral.sh/uv/). The development dependencies are declared as a *dependency group*, so they are installed by `uv` instead of `pip`:

```sh
uv sync
```

This creates `.venv` from the pinned versions in `uv.lock` and installs the default `dev` group, which covers formatting, linting, and testing.

To additionally install the published extras:

```sh
uv sync --all-extras
```

#### Select a PyTorch build

By default, `torch` resolves from PyPI, which serves CUDA-enabled wheels on Linux and CPU-only wheels on macOS and Windows. To choose a specific build, enable one of the accelerator dependency groups, for example the CPU-only one:

```sh
uv sync --group cpu
```

The available groups are `cpu`, `cu126`, `cu128`, and `cu130`. Each points `torch` at the matching [PyTorch index](https://pytorch.org/get-started/locally/), and the default `dev` group is still installed alongside it.

The groups are declared as mutually exclusive, so enable at most one; combining them, including via `uv sync --all-groups`, is rejected. Dependency groups are not published in the package metadata, so `pip install deep-learning-toolkit` is unaffected by this configuration.

After changing dependencies in `pyproject.toml`, refresh and commit the lock file:

```sh
uv lock
```

Continuous integration runs with `UV_LOCKED=1`, so a stale `uv.lock` fails the build. It also syncs the `cpu` group, which keeps the CUDA wheels out of the runner.

### Commands for development

All make targets run their tools through `uv run`:

- `make format`: run `isort` and `black` on `dlk/` and `tests/`
- `make format-check`: check `isort` and `black` formatting without modifying files
- `make lint`: run `basedpyright` on `dlk/` and `tests/`
- `make compile`: run `python -m compileall -q -f` on `dlk/` and `tests/`
- `make test`: run `pytest`
- `make testq`: run `pytest -quiet`
- `make testv`: run `pytest --verbose`
- `make testvv`: run `pytest --verbose --capture=no`

All test targets depend on the `compile` target.

### Building a distribution

```sh
uv build --no-sources
```

---

## Releases

Every released version is listed in [`CHANGELOG.md`](CHANGELOG.md), each entry summarizing what changed and linking to its full notes in [`docs/releases/`](docs/releases/). Those notes add the per-commit changelog and the changed files of the release.

---

## Citing

Citation metadata is provided in [`CITATION.cff`](CITATION.cff), which GitHub renders under "Cite this repository". Releases are archived on [Zenodo](https://zenodo.org/), which mints a DOI for each version.

---

## License

Licensed under the [Apache License 2.0](LICENSE).
