Metadata-Version: 2.5
Name: gpulayer
Version: 0.8.1
Summary: Train PyTorch on every GPU you can reach: one Trainer, from your laptop to many machines. NVIDIA CUDA, Apple Metal and CPU; self-hosted, open source.
Project-URL: Homepage, https://www.gpulayer.dev
Project-URL: Documentation, https://www.gpulayer.dev/docs/
Project-URL: Source, https://github.com/antonellof/gpulayer
Project-URL: Changelog, https://github.com/antonellof/gpulayer/blob/main/CHANGELOG.md
Project-URL: Issues, https://github.com/antonellof/gpulayer/issues
Author: Antonello
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: apple-silicon,cuda,distributed,gpu,mcp,mlops,orchestration,pytorch,scheduler
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: MacOS
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: System :: Distributed Computing
Requires-Python: >=3.10
Requires-Dist: packaging>=23
Requires-Dist: pydantic<3,>=2.5
Provides-Extra: dev
Requires-Dist: mypy>=1.8; extra == 'dev'
Requires-Dist: prometheus-client>=0.20; extra == 'dev'
Requires-Dist: pytest>=7.4; extra == 'dev'
Requires-Dist: ruff>=0.4; extra == 'dev'
Provides-Extra: examples
Requires-Dist: mlx>=0.10; (sys_platform == 'darwin' and platform_machine == 'arm64') and extra == 'examples'
Requires-Dist: numpy>=1.20; extra == 'examples'
Requires-Dist: torch>=2.1; extra == 'examples'
Provides-Extra: gcp
Requires-Dist: google-auth>=2.20; extra == 'gcp'
Requires-Dist: requests>=2.28; extra == 'gcp'
Provides-Extra: torch
Requires-Dist: torch>=2.1; extra == 'torch'
Description-Content-Type: text/markdown

<p align="center">
  <img src="https://raw.githubusercontent.com/antonellof/gpulayer/main/docs/assets/logo.svg" alt="GPUlayer" width="300">
</p>

<p align="center">
  <b>Train PyTorch on every GPU you can reach.</b><br>
  One Trainer, from your laptop to many machines.<br>
  <a href="#install">Install</a> ·
  <a href="#quickstart">Quickstart</a> ·
  <a href="#performance-and-comparison">Performance</a> ·
  <a href="#for-ai-and-coding-agents">AI agents</a> ·
  <a href="#architecture">Architecture</a> ·
  <a href="https://www.gpulayer.dev/docs/">Docs</a> ·
  <a href="https://github.com/antonellof/gpulayer/tree/main/examples">Examples</a> ·
  <a href="https://github.com/antonellof/gpulayer/blob/main/CHANGELOG.md">Changelog</a>
</p>

The GPUlayer `Trainer` runs the same PyTorch script in your process, on every GPU of a
machine, or across many machines: laptops, workstations, servers and cloud VMs. It picks
machines and GPUs that can train the model, starts every process, and brings back logs,
checkpoints and metrics. It uses PyTorch Lightning's names (`LightningModule` hooks, manual
optimization, `DataModule`, callbacks, loggers, profilers, tuner), so most Lightning code moves
with an import change.

NVIDIA CUDA, Apple Metal (MPS) and CPU are validated on hardware. AMD ROCm is implemented and
not yet validated. Docs: [gpulayer.dev/docs](https://www.gpulayer.dev/docs/).

> **v0.8.1.** Validated on a Tesla T4, 4× RTX 3070, 2× A100 (NVLink), an Apple M2 Pro, and
> distributed CPU training on up to 16 Google Cloud machines. NCCL across machines is validated
> on 8 AWS machines with one Tesla T4 each (DDP, `sync_batchnorm`, manual optimization, FSDP). See
> [Status](https://github.com/antonellof/gpulayer/blob/main/docs/project/status.md).

## Install

```bash
pip install gpulayer
```

Check what this machine can train on:

```bash
gpulayer resources detect
```

To train with PyTorch, install it too: `pip install "gpulayer[torch]"`.

<details>
<summary>Other ways to install</summary>

| Method | Command |
|---|---|
| As a standalone CLI with [uv](https://docs.astral.sh/uv/) | `uv tool install gpulayer` |
| Install script (own virtual environment, no sudo) | `curl -fsSL https://raw.githubusercontent.com/antonellof/gpulayer/main/install.sh \| sh` |
| Docker | `docker pull ghcr.io/antonellof/gpulayer:0.8.1` (also `:0.8.1-torch`; see [deploy/docker](https://github.com/antonellof/gpulayer/tree/main/deploy/docker)) |
| Kubernetes (Helm) | `helm install gpulayer deploy/kubernetes/helm/gpulayer -n gpulayer --create-namespace` (see [Kubernetes](https://github.com/antonellof/gpulayer/blob/main/docs/guides/kubernetes.md)) |
| From source, to run the examples | `git clone https://github.com/antonellof/gpulayer && cd gpulayer && pip install -e ".[dev,examples]"` |

More in [Installation](https://github.com/antonellof/gpulayer/blob/main/docs/get-started/installation.md).
</details>

## Quickstart

```python
import gpulayer.pytorch as glt

trainer = glt.Trainer(max_epochs=5, accelerator="cuda", devices=4, num_nodes=2)
trainer.fit(model, train_loader)          # 2 machines x 4 GPUs, DDP over NCCL
```

```bash
# The same file in this process, as 2-process DDP, or on 2 nodes: only the flags change
python examples/trainer_regression/train.py
python examples/trainer_regression/train.py --accelerator cpu --devices 2
python examples/trainer_regression/train.py --accelerator cpu --nodes 2 --devices 2

```

More machines: start a server on one, join the others, and point your laptop at it. Each
line is printed by `gpulayer up`; on a single machine, use three terminals.

```bash
gpulayer up                                                    # machine 1: the server, runs jobs too
gpulayer join https://192.168.1.56:8470 --token glj_... --fingerprint sha256:...   # each GPU machine
export GPULAYER_URL=https://192.168.1.56:8470 GPULAYER_TOKEN=glc_... GPULAYER_FINGERPRINT=sha256:...
python examples/trainer_regression/train.py --accelerator cpu --nodes 2   # one node per worker
```

The [full quickstart](https://github.com/antonellof/gpulayer/blob/main/docs/get-started/quickstart.md) shows the output of each step.

## Performance

Against PyTorch Lightning, with the same model, data and optimizer and each trainer's
defaults, the GPUlayer Trainer was faster in 13 of 17 GPU cases and in every CPU case
(GPUlayer 0.8, 2026-11-01). Lightning was faster on one GPU with a CNN on one host, and by 1 to
9% on a CNN across 2 to 8 machines over NCCL.

![Training step time: GPUlayer and PyTorch Lightning](https://raw.githubusercontent.com/antonellof/gpulayer/main/docs/assets/perf/training-step-light.svg#gh-light-mode-only)
![Training step time: GPUlayer and PyTorch Lightning](https://raw.githubusercontent.com/antonellof/gpulayer/main/docs/assets/perf/training-step-dark.svg#gh-dark-mode-only)
![Several GPUs in one machine](https://raw.githubusercontent.com/antonellof/gpulayer/main/docs/assets/perf/multigpu-step-light.svg#gh-light-mode-only)
![Several GPUs in one machine](https://raw.githubusercontent.com/antonellof/gpulayer/main/docs/assets/perf/multigpu-step-dark.svg#gh-dark-mode-only)
![One GPU per machine, NCCL across machines](https://raw.githubusercontent.com/antonellof/gpulayer/main/docs/assets/perf/nccl-step-light.svg#gh-light-mode-only)
![One GPU per machine, NCCL across machines](https://raw.githubusercontent.com/antonellof/gpulayer/main/docs/assets/perf/nccl-step-dark.svg#gh-dark-mode-only)

| Hardware | GPUlayer Trainer against Lightning |
|---|---|
| Tesla T4 | 1.08 to 1.38x per step |
| 1, 2 and 4 RTX 3070s (DDP over NCCL) | 1.26 to 1.60x in 5 cases; 1.12x slower on one GPU with a CNN |
| 2, 4 and 8 machines with one T4 each (NCCL) | MLP 1.08 to 1.29x faster; CNN 1.01 to 1.09x slower |
| Apple M2 Pro, GPU and CPU | 1.02 to 1.80x |
| 8 CPU machines on AWS (gloo, 4 processes each) | 17.8% faster (Lightning under torchrun) |

Keeping a 4-machine run in one zone instead of two regions made it 1.38x faster (42.5 s vs
58.8 s). Method and logs:
[Performance](https://github.com/antonellof/gpulayer/blob/main/docs/get-started/performance-and-comparison.md);
scripts in [`benchmarks/`](https://github.com/antonellof/gpulayer/blob/main/benchmarks/README.md).

## How it works

![Your training goes through GPUlayer to the machines you can reach](https://raw.githubusercontent.com/antonellof/gpulayer/main/docs/assets/architecture-light.svg#gh-light-mode-only)
![Your training goes through GPUlayer to the machines you can reach](https://raw.githubusercontent.com/antonellof/gpulayer/main/docs/assets/architecture-dark.svg#gh-dark-mode-only)

The Trainer submits a run that says what it needs (accelerator, number of GPUs and machines,
GPU memory). A control plane, inside your process or run as a server with `gpulayer up`,
picks machines that can train it: the best-connected GPUs of a machine, and machines in one
zone when it can. A worker on each machine starts the ranks, streams logs and uploads
checkpoints. Workers open every connection themselves, so machines can sit behind NAT. More
in the [Architecture overview](https://github.com/antonellof/gpulayer/blob/main/docs/architecture/overview.md).

The same engine runs plain commands and Python functions as jobs, and `gpulayer mcp` lets
coding agents start runs and read their results
([AI agents](https://github.com/antonellof/gpulayer/blob/main/docs/guides/ai-agents.md)).

With one GPU on the machine you are using, a plain training loop is simpler. Jobs are not
sandboxed, so use GPUlayer among people who trust each other.

## Development

```bash
uv pip install -e '.[dev,examples]'
.venv/bin/pytest && .venv/bin/ruff check src tests && .venv/bin/mypy src
```

See [Contributing](https://github.com/antonellof/gpulayer/blob/main/docs/project/contributing.md).

## License

Apache License 2.0, see [LICENSE](https://github.com/antonellof/gpulayer/blob/main/LICENSE).
