Metadata-Version: 2.4
Name: deep-learning-toolkit
Version: 0.4.0
Summary: Deep Learning Toolkit — Reusable PyTorch building blocks for artificial intelligence & scientific machine learning: networks, losses, training loops, and utilities.
Author-email: Johann Rudi <jrudi@vt.edu>
License-Expression: GPL-3.0-only
Project-URL: Homepage, https://github.com/johannrudi/deep-learning-toolkit
Project-URL: Repository, https://github.com/johannrudi/deep-learning-toolkit
Project-URL: Issues, https://github.com/johannrudi/deep-learning-toolkit/issues
Keywords: artificial intelligence,deep learning,machine learning,neural networks,scientific computing,pytorch
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: matplotlib<4,>=3
Requires-Dist: prettytable<4,>=3
Requires-Dist: pyyaml<7,>=6
Requires-Dist: torch<3,>=2
Requires-Dist: tqdm<5,>=4
Provides-Extra: dev
Requires-Dist: basedpyright; extra == "dev"
Requires-Dist: black<26,>=25; extra == "dev"
Requires-Dist: build; extra == "dev"
Requires-Dist: bumpver; extra == "dev"
Requires-Dist: isort<6,>=5; extra == "dev"
Requires-Dist: pip-tools; extra == "dev"
Requires-Dist: pytest<9,>=8; extra == "dev"
Requires-Dist: twine; extra == "dev"
Provides-Extra: diffusion
Requires-Dist: flow-matching<2,>=1.0.10; extra == "diffusion"
Provides-Extra: kde
Requires-Dist: torch-kde; extra == "kde"
Provides-Extra: test
Requires-Dist: pytest<9,>=8; extra == "test"
Dynamic: license-file

# Deep Learning Toolkit

<!-- NOTE: github badges work with private repos -->
<!--
[![Format](https://github.com/johannrudi/deep-learning-toolkit/actions/workflows/format.yml/badge.svg)](https://github.com/johannrudi/deep-learning-toolkit/actions/workflows/format.yml)
[![Compile](https://github.com/johannrudi/deep-learning-toolkit/actions/workflows/compile.yml/badge.svg)](https://github.com/johannrudi/deep-learning-toolkit/actions/workflows/compile.yml)
[![Lint](https://github.com/johannrudi/deep-learning-toolkit/actions/workflows/lint.yml/badge.svg)](https://github.com/johannrudi/deep-learning-toolkit/actions/workflows/lint.yml)
[![Test](https://github.com/johannrudi/deep-learning-toolkit/actions/workflows/test.yml/badge.svg)](https://github.com/johannrudi/deep-learning-toolkit/actions/workflows/test.yml)
-->
<!-- NOTE: shields.io badges only work with public repos -->
[![Format](https://img.shields.io/github/actions/workflow/status/johannrudi/deep-learning-toolkit/format.yml?style=for-the-badge&label=Format)](https://github.com/johannrudi/deep-learning-toolkit/actions/workflows/format.yml)
[![Compile](https://img.shields.io/github/actions/workflow/status/johannrudi/deep-learning-toolkit/compile.yml?style=for-the-badge&label=Compile)](https://github.com/johannrudi/deep-learning-toolkit/actions/workflows/compile.yml)
[![Lint](https://img.shields.io/github/actions/workflow/status/johannrudi/deep-learning-toolkit/lint.yml?style=for-the-badge&label=Lint)](https://github.com/johannrudi/deep-learning-toolkit/actions/workflows/lint.yml)
[![Tests](https://img.shields.io/github/actions/workflow/status/johannrudi/deep-learning-toolkit/tests.yml?style=for-the-badge&label=Tests)](https://github.com/johannrudi/deep-learning-toolkit/actions/workflows/tests.yml)

Reusable [PyTorch](https://pytorch.org/) building blocks for artificial intelligence & scientific machine learning:
networks, losses, training loops, and utilities.

---

## Installing the `deep-learning-toolkit`

### Requirements

- Python `>=3.11`

### Runtime dependencies

- `prettytable>=3,<4`
- `pyyaml>=6,<7`
- `torch>=2,<3`
- `tqdm>=4,<5`

### Install in regular mode

```sh
pip install deep-learning-toolkit
```

### Install the package in editable mode

When using a clone of the [Git repository](https://github.com/johannrudi/deep-learning-toolkit/),
run this command from inside the cloned directory:

```sh
pip install -e .
```

#### Install with optional extras

Dependencies for kernel density estimation:

```sh
pip install -e ".[kde]"
```

Dependencies for running tests:

```sh
pip install -e ".[test]"
```

#### Install with optional extras for development

```sh
pip install -e ".[dev]"
```

Using `[dev]` includes extras from `[test]`.

---

## Importing and using `dlk`

To use the toolkit, import its modules in your Python code like this:

```py
from dlk.nets.mlp import MLPNet
from dlk.opt.train import train_epochs

# load your data
...

# create the model
net = MLPNet(input_size=784, output_size=10)

# train the model
train_epochs(n_epochs=100, net=net, dataloader=..., optimizer=..., loss_fn=...)

# evaluate
...
```

---

## Architecture

### Neural network architectures &rarr; `dlk/nets/`

- `mlp.py`: Multilayer Perceptron (MLPNet, MLPNet_MultIn, MLPResNet with residual and attention blocks)
- `autoencoder.py`: Generic autoencoder wrapper for encoder/decoder pairs
- `conv1d.py`, `conv2d.py`: 1D/2D convolutional networks and UNet components (Downsample, Upsample)
- `unet.py`: Complete UNet implementations (older UNet1D/UNet2D and newer UNetXd_2025 architecture)
- `transformer1d.py`: 1D transformer networks with patch embeddings and multi-head attention
- `efficientnet.py`: EfficientNet architecture

#### Network initialization

All network modules follow a consistent pattern:

- Constructor calls `self.init_parameters()` at the end
- `init_parameters()` uses Xavier initialization with gain calculated from activation functions
- Utility functions `_get_gain()` and `_set_init_parameters()` handle activation-aware initialization

### Training and optimization &rarr; `dlk/opt/`

- `train.py`: Training loops (`train_epochs`, `train_batches`) with checkpointing and validation hooks
- `train_gan.py`: GAN-specific training loops
- `scheduler.py`: Learning rate schedulers (multi-stage: linear warmup, constant, cosine annealing)

#### Logging of the training progress

Training functions return detailed logging dictionaries (`dlog`) containing:

- Per-epoch loss statistics (`loss_mean`, `loss_std`)
- Batch-level logs nested in `batch_dlog`
- Total training time in `time_train`
- Checkpointing saves model and optimizer states at specified intervals

### Additional components of the package

- `dlk/mgmt/`: Management of configuration parameter loading/saving, logging, etc.
- `dlk/loss/`: Loss functions
- `dlk/metrics/`: Metrics for evaluating trained nets

---

## Development

### Commands for development

- `make format`: run `isort` and `black` on `dlk/` and `tests/`
- `make format-check`: check `isort` and `black` formatting without modifying files
- `make compile`: run `python -m compileall -q -f` on `dlk/` and `tests/`
- `make lint`: run `basedpyright` on `dlk/` and `tests/`
- `make test`: run `pytest` (after `make compile`)
- `make testq`: run `pytest -q` (after `make compile`)
- `make testv`: run `pytest -v` (after `make compile`)
- `make testvv`: run `pytest -sv` (after `make compile`)
