Metadata-Version: 2.4
Name: verlex
Version: 0.24.0
Summary: One line of Python. Every cloud's best GPU price.
Author: Verlex Team
Maintainer: Verlex Team
License: Apache-2.0
Project-URL: Homepage, https://verlex.dev
Project-URL: Documentation, https://verlex.dev/docs
Keywords: cloud,serverless,machine-learning,gpu,cloud-computing,distributed-computing
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: System :: Distributed Computing
Classifier: Topic :: Internet :: WWW/HTTP :: HTTP Servers
Classifier: Framework :: AsyncIO
Classifier: Typing :: Typed
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: cloudpickle>=3.0.0
Requires-Dist: httpx>=0.25.0
Requires-Dist: pynvml>=11.5.0
Provides-Extra: cli
Requires-Dist: rich>=13.0.0; extra == "cli"
Provides-Extra: mcp
Requires-Dist: mcp<2,>=1.9; extra == "mcp"
Provides-Extra: overflow
Requires-Dist: psutil>=5.9.0; extra == "overflow"
Provides-Extra: agent
Requires-Dist: psutil>=5.9.0; extra == "agent"
Provides-Extra: dev
Requires-Dist: pytest>=7.0.0; extra == "dev"
Requires-Dist: pytest-asyncio>=0.21.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
Requires-Dist: black>=23.0.0; extra == "dev"
Requires-Dist: ruff>=0.1.0; extra == "dev"
Requires-Dist: mypy>=1.0.0; extra == "dev"
Requires-Dist: sentry-sdk>=1.0.0; extra == "dev"
Dynamic: license-file

# Verlex

**One line of Python. Every cloud's best GPU price.**

Verlex reads any Python function, picks the hardware it needs, prices it live across **11 VM clouds**, and runs it on the cheapest one. If a provider fails or stocks out, the job restarts on the next cheapest, automatically. Four additional serverless lanes handle eligible short jobs. No hardware picking or cloud-account setup.

The provider's GPU price passes through **at cost**. Verlex adds a fixed service fee per GPU-hour, prorated per second. That is the whole model: no percentage markup, no hidden margin.

## Installation

```bash
pip install verlex
```

Requires Python 3.10, 3.11, 3.12, or 3.13. Grab your API key from the [dashboard](https://verlex.dev).

## Quick Start

```python
import verlex

# Define your function
def train_model():
    import torch
    model = torch.nn.Linear(100, 10)
    # Your training code here...
    return {"accuracy": 0.95}

# Run it on a cloud GPU - that's it!
result = verlex.cloud(train_model, gpu="A100", api_key="gw_your_key")
print(result)
```

## Basic Usage

### Pass your API key

Every function accepts `api_key` directly, no context manager needed. Set `VERLEX_API_KEY` in your environment to omit it entirely.

```python
import verlex

result = verlex.cloud(my_function, api_key="gw_your_key")
```

### Passing inputs

`args=` holds your function's inputs and `kwargs=` its keyword arguments. Every other keyword configures the job and never reaches your function, so a function parameter called `timeout` or `memory` can never be mistaken for a job option.

```python
# No inputs
verlex.cloud(train_model)

# Positional inputs
verlex.cloud(square, args=[5])
verlex.cloud(train, args=[dataset, epochs])

# Keyword arguments for your function
verlex.cloud(train, args=[dataset], kwargs={"epochs": 5})

# One input that is itself a list
verlex.cloud(total, args=[[1, 2, 3]])
```

Inputs are checked against your function's signature before anything is uploaded or billed. The older form `verlex.cloud(train, [dataset])` still works when unambiguous and warns with the exact rewrite. Each run prints its job id at submit and a summary line at the end; `verlex.last_job()` returns the job id, status, cost, provider, GPU, duration and downloaded outputs.

### Specifying resources

Every hardware field is optional, and **only the fields you set are pinned**. Whatever you leave out is sized from your code and chosen for cost and availability. Asking for a GPU and nothing else is the normal case.

```python
def fit(data):
    ...

# One GPU pinned; cpu, memory and provider are chosen for you
result = verlex.cloud(fit, args=[data], gpu="A100")

# Pin more when you need it; each field is independent
result = verlex.cloud(fit, args=[data], gpu="H100", gpu_count=8)
result = verlex.cloud(fit, args=[data], cpu=4, memory="16 GB")
```

`gpu=` takes one GPU name; leave it unset and Verlex picks one for you. `gpu_count` above 1 puts that many GPUs (up to 8) on one machine and needs the Performance plan; your code uses them itself, for example with `torch.nn.DataParallel`.

Values are checked in your process, before anything is provisioned: a misspelled option answers `did you mean gpu=?`, and `gpu="64 GB"` tells you it belongs in `memory=`.

The full set of job options:

```python
result = verlex.cloud(
    fit,
    args=[data],
    gpu="A100",        # GPU type
    cpu=8,             # vCPU cores
    memory="64GB",     # memory
    provider="aws",    # pin to a cloud (omit to let Verlex pick the cheapest)
    spot=True,         # interruptible capacity
    timeout=7200,      # total seconds from submit to result
    pip_packages=["numpy==1.26.4"],  # extra packages, always installed
    python_version="3.11",  # match your local Python
)
```

Every option, default and environment variable is listed in the [configuration reference](https://github.com/LucasBarcelosGrenier/GateWay/blob/main/docs/CONFIGURATION.md).

## Execution Modes

One flag controls your price-speed tradeoff.

| Mode | Flag | Behavior |
|------|------|----------|
| **Performance** | `fast=True` | Fastest start, at a higher per-GPU-hour fee. Best for time-sensitive workloads. |
| **Standard** | `fast=False` (default) | Picks the cheapest qualified machine. Best for batch jobs and cost-sensitive work. |

```python
# Performance mode, fastest start
result = verlex.cloud(my_function, api_key="gw_your_key", fast=True)

# Standard mode (default), lowest price
result = verlex.cloud(my_function, api_key="gw_your_key")
```

Performance mode (`fast=True`) requires the Performance plan ($10/mo). Standard mode is available on every plan. Both modes share one queue; Performance speeds up machine selection and launch, not queue position. `verlex.estimate_cost(..., fast=True)` prices it before you run.

## Pricing & Billing

Two parts: a monthly plan, then a per-job cost. Verlex passes the provider's live hardware price through at cost and adds a fixed service fee per GPU-hour, prorated per second. There is no percentage markup on the hardware.

The service-fee tier is derived from the hardware's FP16 tensor performance (TFLOPS):

| Tier | Example hardware | Standard (`fast=False`) | Performance (`fast=True`) |
|------|------------------|-------------------------|---------------------------|
| CPU (CPU-only jobs) | No GPU | $0.02/hr | $0.02/hr |
| Small (<185 TFLOPS) | T4, L4, A10, RTX 3090 | $0.10/GPU-hr | $0.20/GPU-hr |
| Mid (<600 TFLOPS) | A100, L40S, RTX 4090 | $0.30/GPU-hr | $0.45/GPU-hr |
| Large (<1500 TFLOPS) | H100, H200, MI300X | $0.40/GPU-hr | $0.60/GPU-hr |
| Flagship (≥1500 TFLOPS) | B200, B300, GB200 | $0.50/GPU-hr | $0.75/GPU-hr |

The service fee is charged per GPU-hour, so an 8-GPU job pays it 8 times. There is no cold-start surcharge; a cold machine and a warm one cost the same.

- **Per-second billing, 1 second minimum.** Both the provider cost and the service fee are prorated to the second. You pay only for the time your job actually runs.
- **Prepaid credits.** Buy credits in the dashboard (minimum top-up $10). Each job places a hold and settles the exact amount when it finishes.
- **Auto top-up (opt-in).** Adds credits with your saved card when the balance runs low (default: $50 added when it drops below $10).
- **Funds-based execution.** There is no maximum job runtime. A job runs until your credits are exhausted, it stops making progress for 30 minutes, or it finishes. The optional `timeout` argument is the total time from submit to result, including queueing and provisioning; when it passes, the SDK cancels the job.

### Plans

| Plan | Price | Includes |
|------|-------|----------|
| **Standard** | $0/mo, free forever | Standard mode, prepaid credits, per-second billing, 25 GB storage |
| **Performance** | $10/mo | Everything in Standard, plus fast mode (`fast=True`) and 300 GB storage |
| **Enterprise** | Custom | Organization teams with a shared credit pool, 1 TB+ storage |

Per-second billing, no credit card required to join.

## Providers, Failover & Serverless

Every job is priced across **11 VM clouds** (AWS, GCP, Azure, Verda, RunPod, Vast.ai, JarvisLabs, Hyperstack, TensorDock, Lyceum, Lambda Labs) plus **four serverless container lanes** (Beam, Northflank, Novita, Cerebrium), and routed to the cheapest machine that fits. Capacity and maintenance state change, so `GET https://api.verlex.dev/v1/providers` is the live source of truth.

```python
import verlex

# Default: priced across 11 VM clouds, routed to the cheapest machine that fits
result = verlex.cloud(train_model, gpu="A100", api_key="gw_your_key")

# Provider list: cost comparison and failover stay inside your subset
result = verlex.cloud(
    train_model,
    gpu="A100",
    provider=["runpod", "gcp", "verda"],
    api_key="gw_your_key",
)

# Hard pin: one provider, no cross-provider failover if it is out of stock
result = verlex.cloud(train_model, gpu="A100", provider="aws", api_key="gw_your_key")
```

- **Automatic failover.** If a launch fails or a region is out of stock, Verlex fails over: nearby regions first, then other providers, then substitute GPUs, always cheapest first. Your job does not fail because one cloud ran out of capacity.
- **Pinning.** A single provider pins hard with no failover. Prefer a provider list, which keeps cost comparison and failover inside your chosen subset.
- **Serverless.** Small jobs route to serverless container lanes (billed per second) when that is cheaper than a VM. This happens automatically; nothing to configure.

## Files & Workspace Sync

Send local files up to the VM before your job runs and bring generated files back when it finishes. Pass a `Workspace` to any cloud call. Auto mode mirrors your project root (respecting `.gitignore`) and returns whatever your code creates or changes; or list explicit `Upload` and `Output` specs for full control.

```python
import verlex
from verlex import Workspace

def train():
    from pathlib import Path
    Path("runs").mkdir(exist_ok=True)
    Path("runs/loss.json").write_text('{"loss": 0.42}')
    Path("model.pt").write_text("weights")
    return {"status": "done"}

# Auto mode mirrors your project root up to the VM, then returns
# new and changed files to the same paths locally
result = verlex.cloud(train, workspace=Workspace(), gpu="A100", api_key="gw_your_key")

# ./runs/loss.json and ./model.pt now exist locally
```

Your function runs inside the synced workspace, so relative paths like `open("data/train.csv")` work unchanged. Import `Workspace`, `Upload`, and `Output` from the top-level `verlex` package.

## Dependencies

Verlex scans your function's source, pins the versions of imported packages installed locally, and bundles local `.py` modules automatically. When you need certainty, pass `pip_packages`: those packages are authoritative, always installed, and a failed install fails the job loudly instead of continuing silently.

```python
result = verlex.cloud(
    train,
    api_key="gw_your_key",
    pip_packages=["torch==2.3.1", "numpy==1.26.4"],
    python_version="3.11",  # 3.10, 3.11, 3.12, or 3.13
)
```

## Pre-warming

Kill the cold start before it happens. `verlex.prewarm(fn)` analyzes your function, boots the right hardware, and hands you back a handle once the machine is ready. Calling the handle runs your function with no provisioning wait, and the machine stays attached between calls, so every call is warm.

```python
import verlex

def train(batch):
    import torch
    ...

wf = verlex.prewarm(train, gpu="A100", api_key="gw_your_key")  # A100 starts booting NOW

data = load_and_clean()      # local preprocessing runs meanwhile

result = wf(data)            # runs train(data) on the warm machine
more = wf(other_data)        # still warm, the VM stays attached

wf.release()                 # or use `with verlex.prewarm(train, gpu="A100") as wf:`
```

## Automatic Cloud Offloading (experimental)

This API is experimental. Don't want to manage when code runs in the cloud? Let Verlex decide. `verlex.overflow()` watches CPU, memory, and GPU. When usage exceeds 85%, heavy functions are transparently offloaded to the cheapest cloud provider; everything else keeps running locally.

```python
import verlex

verlex.overflow(api_key="gw_your_key")

# Your code runs normally.
# When CPU, memory, or GPU exceeds 85%, functions go to the cloud.
data = load_data()
result = train_model(data)   # system overloaded? → cloud
evaluate(result)             # resources free → runs locally
```

Install the monitoring dependency with `pip install 'verlex[overflow]'`.

## Interactive Sessions

`verlex.session()` keeps one VM and model process warm across turns. The
handler loads once, replies stream token by token, and the machine stops when
the context exits or the idle timeout expires.

```python
import verlex
from verlex import serve_loop

def serve():
    model = load_model()

    def reply(prompt, history):
        return model.stream(prompt)

    serve_loop(reply)

with verlex.session(serve, gpu="T4") as session:
    for token in session.send("Explain gradient checkpointing"):
        print(token, end="", flush=True)
```

This API is experimental. Use `verlex chat` for the equivalent CLI workflow.

## AI Assistants (MCP)

Install `verlex[mcp]` and run `verlex mcp serve` to connect an MCP-compatible
assistant. The connector supports jobs and logs, live pricing and billing,
held GPU nodes, file transfer, output artifacts, inference endpoints, and
opt-in team tools. See
[the MCP guide](https://github.com/LucasBarcelosGrenier/GateWay/blob/main/docs/MCP.md)
for client configuration and safety limits.

## Inference Endpoints

Deploy a Hugging Face model or a prior Verlex job output as an always-on
endpoint from Python, the CLI (`verlex deploy create MODEL`), MCP or the
dashboard. Decoder language models expose an OpenAI-compatible API; other
supported tasks expose `/predict` and a self-describing `/schema`. Endpoints are
billed per GPU-second until stopped, not per token, so set `idle_timeout` or
call `stop()`.

```python
import verlex

endpoint = verlex.deploy("Qwen/Qwen2.5-0.5B-Instruct", idle_timeout=1800)
reply = endpoint.chat("Hello!")
print(reply["choices"][0]["message"]["content"])
endpoint.stop()
```

## Checkpoints & Spot Recovery

Spot capacity is 60-80% cheaper than on-demand, but the provider can reclaim it at any time. Anything your job writes into the checkpoint directory (its path is in the `GATEWAY_CHECKPOINT_DIR` environment variable on the machine) syncs to object storage while it runs. When a spot instance is reclaimed, the job is re-enqueued on the next cheapest capacity, the checkpoint directory is restored before your code runs again, and the reclaimed time is not billed. HuggingFace Trainer, PyTorch Lightning, and Keras jobs resume with no code changes.

## Available GPUs

Request a GPU by name (`gpu="A100"`). Verlex finds the cheapest available instance across all 11 VM clouds. Common catalog types include T4, L4, A10, V100, A100, L40, L40S, H100, H200, and B200; additional types appear as providers expose stock. If you don't specify a GPU, Verlex auto-detects your code's needs and picks the best option. `GET https://api.verlex.dev/v1/hardware/gpu-types` returns the currently schedulable public catalog.

## Authentication

```python
import verlex

# Pass api_key directly to any function
result = verlex.cloud(my_function, api_key="gw_your_key")

# Or use the environment variable (VERLEX_API_KEY)
result = verlex.cloud(my_function)  # picks up from env
```

The key is looked up in this order: `api_key=`, `verlex.auth()`, `VERLEX_API_KEY`, `GATEWAY_API_KEY`, then the key saved by `verlex login`.

Both `gw_live_` and `gw_test_` prefixes only name the key type; every key runs against the live service with live billing.

## Error Handling

Verlex raises specific exceptions so you can handle failures gracefully:

```python
from verlex import (
    VerlexError,              # Base class for all Verlex errors
    ConfigurationError,       # Bad input or option (also a ValueError), raised before submit
    AuthenticationError,      # Missing, invalid, frozen or paused credentials
    InsufficientCreditsError, # Not enough credits, or no payment method
    PlanUpgradeRequiredError, # Feature needs the Performance plan
    GPUUnavailableError,      # Pinned hardware has no capacity (alternatives are printed)
    ProviderMaintenanceError, # Pinned provider is under maintenance
    JobFailedError,           # Your code or the platform failed the job
    JobCancelledError,        # The job was cancelled elsewhere
    JobTimeoutError,          # Your timeout passed
    OutputDownloadError,      # A required output could not be downloaded
    SerializationError,       # Function or result could not be (de)serialized
    NetworkError,             # Connection to Verlex API failed
    RateLimitError,           # Too many requests
)
```

An exception raised by your function keeps its type:

```python
try:
    verlex.cloud(train, args=[data])
except ValueError as e:        # a ValueError raised on the machine
    print(e.remote_type, e.job_id)
```

Every error, its cause and its fix are listed in the [errors guide](https://github.com/LucasBarcelosGrenier/GateWay/blob/main/docs/ERRORS.md).

## Links

- **Website**: [verlex.dev](https://verlex.dev)
- **Documentation**: [verlex.dev/docs](https://verlex.dev/docs)

## Contact

- **Support**: support@verlex.dev
- **Sales**: sales@verlex.dev
- **General**: contact@verlex.dev

## License

Apache 2.0
