Metadata-Version: 2.4
Name: vaultlayer
Version: 0.1.84
Summary: Long GPU training runs that finish: automatic checkpointing, recovery and resume on rented GPUs
Author: VaultLayer
License: MIT
Keywords: gpu,cloud,training,checkpointing,fault-tolerance,mlops,ai
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: System :: Distributed Computing
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: click>=8.1.0
Requires-Dist: httpx>=0.27.0
Requires-Dist: pydantic>=2.7.0
Requires-Dist: python-dotenv>=1.0.0
Requires-Dist: redis>=5.0.0
Requires-Dist: aiohttp>=3.9.0
Requires-Dist: boto3>=1.35.0
Requires-Dist: anthropic>=0.40.0
Requires-Dist: huggingface-hub>=0.20.0
Requires-Dist: requests>=2.31.0
Requires-Dist: pyyaml>=6.0
Provides-Extra: server
Requires-Dist: fastapi<0.138,>=0.115.0; extra == "server"
Requires-Dist: uvicorn[standard]>=0.29.0; extra == "server"
Requires-Dist: supabase>=2.4.0; extra == "server"
Requires-Dist: resend>=2.0.0; extra == "server"
Requires-Dist: stripe>=9.0.0; extra == "server"
Requires-Dist: pydantic[email]>=2.7.0; extra == "server"
Requires-Dist: python-multipart>=0.0.9; extra == "server"
Requires-Dist: google-cloud-storage>=2.16.0; extra == "server"
Requires-Dist: azure-storage-blob>=12.20.0; extra == "server"

# VaultLayer

Your long GPU training run finishes, no matter what kills it. VaultLayer
checkpoints your job to object storage, watches the machine, and when it dies,
resumes from the last good checkpoint on fresh capacity.

```bash
pip install -U vaultlayer
vl init
vl run train.py
```

```
Job submitted
Training on vast_ai
Training output:
...
Training completed successfully.
```

---

## What It Does

VaultLayer sits between your training script and the GPUs. It:

- **Checkpoints on a schedule** — saves model and optimizer state to Cloudflare R2 on a timer you can tune, plus an emergency save when the provider sends a termination notice
- **Detects a dead or dying machine** — watches for lost heartbeats and provider termination notices, and fences the old instance so it can't overwrite the new one
- **Resumes on fresh capacity** — provisions a replacement on the same provider or a different one and resumes from the last good checkpoint
- **Bills from one prepaid account** — provider cost plus a flat platform fee, across the providers below

No checkpoint code to write. Hugging Face Trainer scripts work as they are; for a
plain PyTorch or JAX training loop, `vl run` proposes a small checkpoint
integration, shows you the diff, and asks before changing anything. No YAML configs.

### Commands

```bash
# Training
vl run train.py
vl ps
vl logs <job-id> --follow
vl stop <job-id>

# Dataset storage (no S3 required)
vl sync ./data --dataset-id my-dataset
vl run --data r2://my-dataset train.py
vl datasets
```

---

## Supported Providers

| Provider | Status |
|---|---|
| Vast.ai | Available |
| RunPod | Available |
| Lambda Labs | Available |
| AWS (Spot and On-Demand) | Available |
| Nebius (EU), Voltage Park, Hyperstack | In testing |
| GCP, CoreWeave, Crusoe, Azure | Not yet available |

Generally available for single-GPU jobs today; multi-GPU is in limited testing.

---

## Model Size Support

| Model Size | Method | Checkpoint Size | Status |
|---|---|---|---|
| 1B | QLoRA | small | Validated smoke path |
| 3B | QLoRA | small | Validated matrix path |
| 7B | QLoRA | medium | Validated matrix path |
| 72B | QLoRA | large | Validated: a 28 GPU-hour single-GPU run completed |
| Full fine-tune / multi-GPU | varies | varies | Future work |

---

## SLA

VaultLayer tracks job completion, checkpoint persistence, and resume behavior.
Public SLA numbers are not committed during beta.

---

## Dataset Storage (No S3 Required)

VaultLayer's Neutral Zone (Cloudflare R2) is a first-class storage provider. Users with no AWS or
cloud storage account can upload training data directly and train from it on any provider.

```bash
# Upload from your laptop / on-prem server
vl sync ./training-data --dataset-id my-dataset

# Train — data is unpacked to $VAULTLAYER_LOCAL_DATA (./data beside your script)
vl run --data r2://my-dataset train.py

# See what you're storing and the monthly cost
vl datasets
```

**Pricing:**

| Action | Cost |
|--------|------|
| Upload (local → R2) | Free |
| Storage | $0.020 / GB / month ($0.0195 — 30% markup over Cloudflare R2 base rate) |
| Read (R2 → training node) | $0.00 (zero egress within Cloudflare network) |
| S3 mirror (one-time) | AWS egress charge (~$0.09/GB, first 100 GB/month free) |

**Storage quotas by plan:**

| Plan | Storage limit |
|------|--------------|
| Free | 10 GB |
| Pro | 500 GB |
| Enterprise | Unlimited |

Datasets are soft-deleted with `vl datasets --delete <id>` — billing stops immediately,
R2 objects are purged within 24 hours.

## Getting Started

```bash
pip install -U vaultlayer
vl init
vl run train.py
```

---

## License

Private — © 2026 VaultLayer
