Metadata-Version: 2.5
Name: gpulayer
Version: 0.7.0
Summary: Train PyTorch on every GPU you can reach: one Trainer, from your laptop to many machines, faster than Lightning. NVIDIA CUDA, Apple Metal and CPU.
Project-URL: Homepage, https://www.gpulayer.dev
Project-URL: Documentation, https://www.gpulayer.dev/docs/
Project-URL: Source, https://github.com/antonellof/gpulayer
Project-URL: Changelog, https://github.com/antonellof/gpulayer/blob/main/CHANGELOG.md
Project-URL: Issues, https://github.com/antonellof/gpulayer/issues
Author: Antonello
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: apple-silicon,cuda,distributed,gpu,mcp,mlops,orchestration,pytorch,scheduler
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: MacOS
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: System :: Distributed Computing
Requires-Python: >=3.10
Requires-Dist: packaging>=23
Requires-Dist: pydantic<3,>=2.5
Provides-Extra: dev
Requires-Dist: mypy>=1.8; extra == 'dev'
Requires-Dist: prometheus-client>=0.20; extra == 'dev'
Requires-Dist: pytest>=7.4; extra == 'dev'
Requires-Dist: ruff>=0.4; extra == 'dev'
Provides-Extra: examples
Requires-Dist: mlx>=0.10; (sys_platform == 'darwin' and platform_machine == 'arm64') and extra == 'examples'
Requires-Dist: numpy>=1.20; extra == 'examples'
Requires-Dist: torch>=2.1; extra == 'examples'
Provides-Extra: gcp
Requires-Dist: google-auth>=2.20; extra == 'gcp'
Requires-Dist: requests>=2.28; extra == 'gcp'
Provides-Extra: torch
Requires-Dist: torch>=2.1; extra == 'torch'
Description-Content-Type: text/markdown

<p align="center">
  <img src="https://raw.githubusercontent.com/antonellof/gpulayer/main/docs/assets/logo.svg" alt="GPUlayer" width="300">
</p>

<p align="center">
  <b>Train PyTorch on every GPU you can reach.</b><br>
  One Trainer, from your laptop to many machines, faster than Lightning.<br>
  <a href="#install">Install</a> ·
  <a href="#quickstart">Quickstart</a> ·
  <a href="#performance-and-comparison">Performance</a> ·
  <a href="#why-gpulayer">Why GPUlayer</a> ·
  <a href="#for-ai-and-coding-agents">AI agents</a> ·
  <a href="#architecture">Architecture</a> ·
  <a href="https://www.gpulayer.dev/docs/">Docs</a> ·
  <a href="https://github.com/antonellof/gpulayer/tree/main/examples">Examples</a> ·
  <a href="https://github.com/antonellof/gpulayer/blob/main/CHANGELOG.md">Changelog</a>
</p>

GPUlayer trains PyTorch models with one `Trainer` that runs the same script in your process,
on several GPUs of one machine, or across many machines: your laptop, workstations, servers,
clusters and cloud VMs. It picks machines and GPUs that can run the job, keeps a distributed
group on the best-connected GPUs and in one zone, starts every process, and brings back
logs, checkpoints and metrics. Against PyTorch Lightning, with the same model, data and
optimizer, it was faster in 10 of 11 GPU benchmarks (NVIDIA and Apple) and in every CPU
benchmark.

NVIDIA CUDA, Apple Metal (MPS) and CPU are validated; AMD ROCm is implemented and awaiting
validation on AMD hardware. Under the Trainer is a general job engine, so commands and
Python functions in any language or library run the same way.
Documentation: [gpulayer.dev/docs](https://www.gpulayer.dev/docs/).

> **v0.7.0.** Tested on CPU, Apple Silicon (PyTorch MPS and MLX) and an
> NVIDIA Tesla T4, on machines with several GPUs (4× RTX 3070, 2× A100 with NVLink: NCCL across all
> GPUs of a machine), and on up to 30 machines on Google Cloud. NCCL across machines is implemented
> but not validated yet; cloud provisioning covers Google Cloud. See
> [Status and roadmap](https://github.com/antonellof/gpulayer/blob/main/docs/project/status.md).

## Install

```bash
pip install gpulayer
```

That's it. Check what this machine can run:

```bash
gpulayer resources detect
```

To train with PyTorch, install it too: `pip install "gpulayer[torch]"`.

<details>
<summary>Other ways to install</summary>

| Method | Command |
|---|---|
| As a standalone CLI with [uv](https://docs.astral.sh/uv/) | `uv tool install gpulayer` |
| Install script (own virtual environment, no sudo) | `curl -fsSL https://raw.githubusercontent.com/antonellof/gpulayer/main/install.sh \| sh` |
| Docker | `docker pull ghcr.io/antonellof/gpulayer:0.7.0` (also `:0.7.0-torch`; see [deploy/docker](https://github.com/antonellof/gpulayer/tree/main/deploy/docker)) |
| Kubernetes (Helm) | `helm install gpulayer deploy/kubernetes/helm/gpulayer -n gpulayer --create-namespace` (see [Kubernetes](https://github.com/antonellof/gpulayer/blob/main/docs/guides/kubernetes.md)) |
| From source, to run the examples | `git clone https://github.com/antonellof/gpulayer && cd gpulayer && pip install -e ".[dev,examples]"` |

More in [Installation](https://github.com/antonellof/gpulayer/blob/main/docs/get-started/installation.md).
</details>

## Quickstart

```python
import gpulayer.pytorch as glt

trainer = glt.Trainer(max_epochs=5, accelerator="cuda", devices=4, num_nodes=2)
trainer.fit(model, train_loader)          # 2 machines x 4 GPUs, DDP over NCCL
```

```bash
# The same script in this process, as 2-process DDP, or on 2 nodes: only the flags change
python examples/trainer_regression/train.py
python examples/trainer_regression/train.py --accelerator cpu --devices 2
python examples/trainer_regression/train.py --accelerator cpu --nodes 2 --devices 2

# More machines: run this on one of them, then paste the printed `gpulayer join` line on the others
gpulayer up

# See what ran
gpulayer dashboard
```

Not only PyTorch: any command or Python function runs the same way.

```bash
gpulayer submit --accelerator cuda --source ./app -- ./render.sh
```

![GPUlayer dashboard](https://raw.githubusercontent.com/antonellof/gpulayer/main/docs/assets/dashboard/overview-dark.png)

The [full quickstart](https://github.com/antonellof/gpulayer/blob/main/docs/get-started/quickstart.md) walks through each step with its output.

## Performance and comparison

Measured on NVIDIA GPUs against PyTorch Lightning, with the same model, data and optimizer
and each trainer's defaults:

![Training step time: GPUlayer and PyTorch Lightning](https://raw.githubusercontent.com/antonellof/gpulayer/main/docs/assets/perf/training-step-light.svg#gh-light-mode-only)
![Training step time: GPUlayer and PyTorch Lightning](https://raw.githubusercontent.com/antonellof/gpulayer/main/docs/assets/perf/training-step-dark.svg#gh-dark-mode-only)
![Several GPUs in one machine](https://raw.githubusercontent.com/antonellof/gpulayer/main/docs/assets/perf/multigpu-step-light.svg#gh-light-mode-only)
![Several GPUs in one machine](https://raw.githubusercontent.com/antonellof/gpulayer/main/docs/assets/perf/multigpu-step-dark.svg#gh-dark-mode-only)

- One Tesla T4: GPUlayer's Trainer is 1.25 to 1.39x faster per step on small models and 1.14x
  on a compute-bound CNN, and starts 0.1 to 0.7 s sooner.
- 1, 2 and 4 RTX 3070s (DDP over NCCL): GPUlayer is faster in 5 of 6 cases, by 1.11 to 1.76x,
  with the lead growing with the number of GPUs; the 1-GPU CNN was 1.14x faster with Lightning
  on that host.
- NCCL across 4 GPUs and over NVLink (2x A100), and the best-connected GPU choice, validated
  on rented machines.
- One training job across machines on CPUs (gloo): GPUlayer trains 11.9% faster than
  Lightning under torchrun on 8 machines (50.5 s vs 57.3 s; 5.3% on 4, 1.6% on 2), and it
  starts every machine from one `Trainer(num_nodes=N)` call.
- Placement: the same 4-machine job took 42.5 s when GPUlayer kept it in one zone and 58.8 s
  when it was spread across two GCP regions (median of five, placed by the scheduler).
- Distributed jobs start 1.1 to 2.3 s faster than through torchrun; tested on up to 16
  machines (32 processes) and 30 workers across two GCP regions.
- Across two machines: a distributed job spanning both, fail-fast in 2.7 s, and 55.6 small
  jobs per second.

![One training job on more machines](https://raw.githubusercontent.com/antonellof/gpulayer/main/docs/assets/perf/strong-scaling-light.svg#gh-light-mode-only)
![One training job on more machines](https://raw.githubusercontent.com/antonellof/gpulayer/main/docs/assets/perf/strong-scaling-dark.svg#gh-dark-mode-only)

Method, all charts and raw logs:
[Performance and comparison](https://github.com/antonellof/gpulayer/blob/main/docs/get-started/performance-and-comparison.md). Reproduce them
with [`benchmarks/`](https://github.com/antonellof/gpulayer/blob/main/benchmarks/README.md).

## Why GPUlayer

Most GPU code assumes the GPU is on the machine where you run it. With GPUlayer you describe
what a workload needs (an accelerator, GPU memory, frameworks) and it runs on a machine that
has it, wherever that machine is.

| Situation | Without GPUlayer | With GPUlayer |
|---|---|---|
| Code on a laptop, GPU elsewhere | ssh, copy code, rebuild the environment, tmux, copy results back | `gpulayer submit --target remote -- <cmd>` |
| Several GPU machines | track free machines by hand; jobs crash on the wrong GPU or library | one queue; each job lands where it can run, or is rejected with the reason |
| NVIDIA servers and Apple Silicon Macs | separate scripts and setup per machine | ask for `cuda` or `mps`; one interface |
| Many independent tasks | loops and ad hoc scripts | `fn.map([...])` across free GPUs |
| Multi-node training | `torchrun` on every node with matching ranks and addresses | `Trainer(num_nodes=2, devices=4)`; the machines are kept in one zone or rack when possible |

On one machine with one GPU, running your code directly is simpler. GPUlayer runs services
without a load balancer in front of them, and is not a sandbox for untrusted code. More in
[Why GPUlayer](https://www.gpulayer.dev/docs/#why-gpulayer).

## For AI and coding agents

`gpulayer mcp` is an MCP server, so Claude Code, Claude Desktop, Cursor, Codex and VS Code
agents can list GPUs, submit jobs, wait, read logs and download outputs. An
[Agent Skill](https://github.com/antonellof/gpulayer/blob/main/integrations/skills/gpulayer/SKILL.md) covers agents that use the CLI.

```bash
claude mcp add gpulayer -- gpulayer mcp
```

See [AI and coding agents](https://github.com/antonellof/gpulayer/blob/main/docs/guides/ai-agents.md).

## Architecture

![Your workloads go through GPUlayer to the machines you can reach](https://raw.githubusercontent.com/antonellof/gpulayer/main/docs/assets/architecture-light.svg#gh-light-mode-only)
![Your workloads go through GPUlayer to the machines you can reach](https://raw.githubusercontent.com/antonellof/gpulayer/main/docs/assets/architecture-dark.svg#gh-dark-mode-only)

GPUlayer has three parts:

- **Clients** submit work: the `gpulayer` CLI, the Python SDK (`@gl.remote`, `gl.Client`), the
  PyTorch `Trainer` and `launch()`, and the MCP server for AI agents. A job states what it
  needs (accelerator, GPU count and memory, frameworks), not which machine to use.
- **The control plane** keeps the job queue, picks a worker for each job and records the
  reason when no worker fits. Distributed groups stay in one network group, zone or region
  when possible, and multi-GPU jobs get the best-connected GPUs on a machine. It stores code bundles, inputs, outputs and logs, and checks
  tokens. For work on one machine it runs inside your process; for several machines it runs
  as a server (`gpulayer up`).
- **Workers**, one per machine, report their GPUs and installed frameworks, ask the control
  plane for tasks, run them as normal processes (one per rank for distributed jobs),
  stream logs back and upload outputs. Workers open every connection themselves, so they can
  sit behind NAT. A cloud VM is a worker that joined with `gpulayer join`.

A job goes from `queued` to `scheduled` when a worker fits, then `running`, and ends as
`succeeded`, `failed`, `cancelled` or `timed_out`. Frameworks such as PyTorch, JAX, MLX and
CuPy still decide how the computation uses the GPU; GPUlayer decides which GPU runs it. More in
[Architecture overview](https://github.com/antonellof/gpulayer/blob/main/docs/architecture/overview.md).

## Development

```bash
uv pip install -e '.[dev,examples]'
.venv/bin/pytest && .venv/bin/ruff check src tests && .venv/bin/mypy src
```

See [Contributing](https://github.com/antonellof/gpulayer/blob/main/docs/project/contributing.md).

## License

Apache License 2.0, see [LICENSE](https://github.com/antonellof/gpulayer/blob/main/LICENSE).
