Metadata-Version: 2.1
Name: flow-compute
Version: 3.28.2
Summary: CLI and SDK for submitting and managing GPU workloads
Keywords: gpu,compute,cloud,foundry,ml,ai
Author-Email: "Foundry Technologies, Inc." <support@mithril.ai>
License: Apache-2.0
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Project-URL: Homepage, https://mithril.ai
Project-URL: Documentation, https://github.com/mithrilcompute/flow#readme
Project-URL: Repository, https://github.com/mithrilcompute/flow
Project-URL: Issues, https://github.com/mithrilcompute/flow/issues
Requires-Python: >=3.10
Requires-Dist: pydantic<3.0,>=2.7
Requires-Dist: pyyaml<7.0,>=6.0
Requires-Dist: httpx<1.0,>=0.28.0
Requires-Dist: anyio>=4.14.2
Requires-Dist: jinja2<4.0,>=3.0
Requires-Dist: rich<14.0,>=13.0
Requires-Dist: cloudpickle<4.0,>=3.0
Requires-Dist: click<8.2.0,>=8.1.8
Requires-Dist: prompt-toolkit<4.0,>=3.0.9
Requires-Dist: hishel<1.0.0,>=0.1.3
Requires-Dist: packaging>=22.0
Provides-Extra: s3
Requires-Dist: boto3<2.0,>=1.26; extra == "s3"
Provides-Extra: recommended
Requires-Dist: numpy>=1.20; extra == "recommended"
Requires-Dist: trogon<1.0,>=0.6; extra == "recommended"
Provides-Extra: jupyter
Requires-Dist: ipykernel<7,>=6.29; extra == "jupyter"
Requires-Dist: jupyter-client<9,>=8.6; extra == "jupyter"
Provides-Extra: serverless
Requires-Dist: temporalio<2.0,>=1.7.0; extra == "serverless"
Requires-Dist: starlette<2.0,>=1.0.1; extra == "serverless"
Requires-Dist: uvicorn<1.0,>=0.23.0; extra == "serverless"
Requires-Dist: kubernetes<35.0,>=29.0; extra == "serverless"
Provides-Extra: skypilot
Requires-Dist: skypilot[aws,gcp]<1.0,>=0.11.0; extra == "skypilot"
Description-Content-Type: text/markdown

# Flow CLI & SDK

**Run GPU workloads on Mithril from the shell or from Python.**
Flow procures GPUs through Mithril, spins up InfiniBand-connected instances, and runs your
workloads. It is one engine with two projections: the CLI is what you meet first, and
`import flow` is the same engine for programs. Every CLI verb runs on the SDK.

![PyPI - Version](https://img.shields.io/pypi/v/flow-compute) [Public repo](https://github.com/mithrilcompute/flow)

## First hour

The same three commands `flow --help` teaches:

```bash
uv tool install flow-compute   # or: pipx install flow-compute
flow setup                     # Mithril account, API key and project
flow tutorial                  # Guided first GPU run with health checks
flow run 'nvidia-smi'          # Run a command on a GPU
```

`flow run` without `-i` bids on the cheapest NVIDIA GPU instance type with live capacity,
prints the choice with its family and hourly price, and asks before any bid above $20/hour.
The implicit default never picks a TPU or an accelerator family Flow does not know, since
the first command is `nvidia-smi`; name other hardware with `-i` (`-i v6e-1`, `-i cpu`).
`flow tutorial` prices its GPU test the same way and asks before launching it; `--yes` is
the only scripted consent and `--skip-example` leaves the test out. Add `--dry-run` to see
the configuration without bidding:

```
$ flow run 'nvidia-smi' --dry-run
╭─ Task Configuration ─────────────────────────────────────────────╮
│ Name           run-0972ae                                        │
│ Command        nvidia-smi                                        │
│ Instance Type  a100                                              │
│ Market Price   $8.00/hr (a100, cheapest live NVIDIA GPU)         │
│ Priority       Med                                               │
│ Max Bid        $16.00/hr                                         │
│ Code Upload    auto (timeout 600s) → /workspace (host+container) │
╰──────────────────────────────────────────────────────────────────╯

✓ Configuration is valid
```

Then watch, inspect and stop what you started:

```bash
flow status              # List and monitor tasks
flow logs <task> -f      # Stream task logs
flow ssh <task>          # SSH into a running task
flow cancel <task>       # Cancel a task
```

```
$ flow status
╭─ ❊ Flow • 9 completed ────────────────────────────────────────────╮
│                                                                   │
│     #  Status        Task                          Type           │
│     1  ✓ completed   cli-welcome                   1×A100         │
│     2  ✓ completed   post-training-sanity-1        1×A100·40G     │
│     3  ✓ completed   pretraining-pending-a100-40g  1×A100·40G     │
│     4  ✓ completed   post-training-fine-tune       8×H100         │
│     5  ✓ completed   inference-serving             1×H100         │
│     6  ✓ completed   pretraining-train-8xa100      8×A100         │
│     7  ✓ completed   post-training-sanity-2        1×A100·40G     │
│     8  ✓ completed   reasoning-rollout-b200        8×B200         │
│     9  ✓ completed   pretraining-2048-h100         256×8×H100     │
│                                                                   │
╰───────────────────────────────────────────────────────────────────╯
```

The same first hour from Python:

```python
import flow

task = flow.run("nvidia-smi", instance_type="a100")
for line in task.log_stream():
    print(line, end="")
```

## Use flow with a coding agent

Paste this into Claude Code, Codex or any other coding agent before it runs Flow for you:

<!-- generated:agent-prompt -->
```text
You are operating Flow, the `flow` CLI that runs GPU workloads on Mithril. Follow these rules.

Setup
- Install with `uv tool install flow-compute`.
- The user creates an API key at console.mithril.ai and exports `MITHRIL_API_KEY`; never print
  it. If it is missing, ask the user to set it or to run `flow setup` in their own terminal.
- Ask the user which project to bill. Pass it as `--project <name or id>` before the command on
  every call (`flow --project research status --json`), or export `MITHRIL_PROJECT` once.
- Export `FLOW_NO_INTERACTIVE=1`, keep `FLOW_FORCE_INTERACTIVE` unset and never pass an
  interactive flag, so a command that would prompt fails instead of waiting.

Reading the CLI
- Learn commands, options, capabilities and exit codes from `flow schema --json`, not help
  text. Never use what it marks `"deprecated": true`; use the `replacement` it names.
- Pass `--json` to every command that has the `json` capability. stdout is then one envelope,
  `{"data": ..., "error": null}` on success; on failure `error` carries `code`, `message` and
  `hint`. Check `error` first, since `data` can still hold results. With `--follow`, or on a
  command whose `json_modes` is only `stream`, stdout is one JSON object per line instead.
- Retry only `TRANSIENT` (exit 6), with backoff. `AUTH_REQUIRED` (exit 2): ask the user to fix
  the credentials. Any other `error.code`: change the request or report it; never rerun it as is.
- `APPROVAL_PENDING` (exit 7): it waits for the user's approval. Tell them, stop, never resend it.

Spending
- Commands that create tasks, VMs, bids or reservations spend money. Before each, tell the user
  the project, instance type and count, price cap and expected run time; wait for an explicit yes.
- Preview at no cost: `flow run 'python train.py' -i 8xh100 -m 20 --dry-run --json`. `-m` caps
  the bid per instance-hour; spot bills the market's clearing price, never more than the cap.
- An envelope's `project` names the project its command acted in. Before a spend, check that the
  preview's `project` is the one the user approved; if it is not, stop and ask.
- `--yes` is the only consent the CLI accepts. Pass it only for a spend the user approved;
  `CONFIRMATION_REQUIRED` (exit 3) means the CLI wants the user's decision, not yours.
- Pass `--idempotency-key <key>` to `flow run` so a retry returns the first task, not a second.
- A spot task keeps its machine, and its bill, after its command exits. Once its work is done,
  cancel what you started: `flow cancel <task> --yes --json`. A reservation bills until its end
  time and cannot be ended early.

Operating
- `flow status --json` lists tasks and `flow logs <task> --json` shows output.
  `flow ssh <task> -- <command>` runs a command on the machine and takes no `--json`.
- Full command index, SDK reference and recipes: `flow setup --claude` or `flow setup --codex`.
```
<!-- /generated:agent-prompt -->

---

## Background

> **There's a paradox in GPU infrastructure today:**
> Massive GPU capacity sits idle, even as AI teams wait in queues—starved for compute.
> Mithril, the AI-compute **omnicloud**, dynamically allocates GPU resources from a global pool (spanning Mithril's first-party resources and 3rd-party partner cloud capacity) using efficient two-sided auctions, maximizing surplus and reducing costs. Mithril seamlessly supports both reserved-in-advance and just-in-time workloads—maximizing utilization, ensuring availability, and significantly reducing costs.

---

## Why choose Flow

Status quo GPU provisioning involves quotas, complex setups, and queue delays, even as GPUs sit idle elsewhere or in recovery processes. Flow addresses this:

**Dynamic Market Allocation** – Efficient two-sided auctions ensure you pay the lowest market-driven prices rather than inflated rates.

**Simplified Batch Execution** – An intuitive interface designed for cost-effective, high-performance batch workloads without complex infrastructure management.

Provision from 1 to thousands of GPUs for long-term reservations, short-term "micro-reservations" (minutes to weeks), or spot/on-demand needs—all interconnected via InfiniBand. High-performance persistent storage and built-in Docker support further streamline workloads, ensuring rapid data access and reproducibility.

---

## Why Flow + Mithril?

| Pillar                                              | Outcome                                                                      | How                                                                                  |
| --------------------------------------------------- | ---------------------------------------------------------------------------- | ------------------------------------------------------------------------------------ |
| **Iteration Velocity and Ease**                     | Fresh containers quickly once capacity is allocated.                         | `flow dev` for a GPU workstation or `flow run` to launch tasks                      |
| **Best price-performance via market-based pricing** | Preemptible secure jobs for **\$1/h100-hr**                                  | Blind two-sided second-price auction; client-side bid capping                        |
| **Availability and Elasticity**                     | Self-serve access to spot and reserved capacity, subject to account access and market availability. | Spot capacity plus overflow capacity from partner clouds                             |
| **Abstraction and Simplification**                  | InfiniBand VMs, CUDA drivers, auto-managed healing buffer—all pre-arranged.  | Mithril virtualization and base images preconfigured + Mithril capacity management.  |

> *"The tremendous demand for AI compute and the large fraction of idle time makes sharing a perfect solution, and Mithril's innovative market is the right approach."* — **Paul Milgrom**, Nobel Laureate (Auction Theory and Mechanism Design)

---

## Workflows

### Research mode

* `flow run` → reproducible batch jobs from a command, a YAML file or a SLURM script.
* `flow dev` → interactive development on an allocated GPU workstation.
* `import flow` → the same engine for pipelines and orchestration.

```bash
# Launch a batch job on discounted H100s
flow run 'python train.py' -i 8xh100

# Run a saved configuration
flow run task.yaml

# Frictionlessly leverage an existing SLURM script
flow run job.slurm
```

```python
# Serverless-style decorator
@app.function(instance_type="a100")
def train(lr: float) -> dict:
    ...
```

### Infrastructure mode

* `flow run 'python train.py' -i 8xh100 -N 20` → run on a 20-node GPU cluster
* `flow volume create -s 10000 --interface file` → provision 10 TB of persistent, high-speed storage
* `flow ssh instance -- nvidia-smi` → run across all nodes in parallel

```
$ flow run 'python train.py' -i 8xh100 -N 20 --dry-run
╭─ Task Configuration ────────────────────────────────────────────╮
│ Name          flow-task-0fd2a7                                  │
│ Command       python train.py                                   │
│ Instances     20 × 8xh100                                       │
│ Market Price  $192.00/hr (h100)                                 │
│ Priority      Med                                               │
│ Max Bid       $384.00/hr per instance ($7680.00/hr total)       │
│ Code Upload   auto (timeout 600s) → /workspace (host+container) │
╰─────────────────────────────────────────────────────────────────╯

✓ Dry run passed; nothing was submitted
```

---

## Ideal Use Cases

* **Rapid Experimentation** – Quick iterations for research sprints.
* **Instant Elasticity** – Scale rapidly from one to thousands of GPUs.
* **Collaborative Research** – Shared dev environments with per-task cost controls.

Flow is not yet ideal for: always‑on ≤100 ms inference, strictly on‑prem regulated data, or models that fit on laptop or consumer-grade GPUs.

---

## Architecture (30‑s view)

```
Your intent ⟶ Flow Execution Layer ⟶ Global GPU Fabric
```

*Flow abstracts complex GPU auctions, InfiniBand clusters, and multi-cloud management into a single seamless and unified developer interface.*

---

## Installation

### Requirements

- Python 3.10 or later
- Recommended: use `uv` to auto-manage a compatible Python when installing the CLI
- Linux or macOS for native CLI workflows. On Windows, use WSL2; native Windows is not fully validated.

### 1) Install uv — optional but recommended
Installation guide: [docs.astral.sh/uv/getting-started/installation](https://docs.astral.sh/uv/getting-started/installation/)

- macOS/Linux:
  ```bash
  curl -LsSf https://astral.sh/uv/install.sh | sh
  ```
- Windows: use WSL2 for the Flow CLI, then run the macOS/Linux command inside WSL.
  ```bash
  curl -LsSf https://astral.sh/uv/install.sh | sh
  ```

### 2) Install Flow

- Global CLI (uv):
  ```bash
  uv tool install flow-compute
  flow setup
  ```

- Global CLI (pipx):
  ```bash
  pipx install flow-compute
  flow setup
  ```

### Providers

Flow talks to one provider per configuration. Mithril is the default and the only provider
installed by the base package; `flow setup --provider` (or `FLOW_PROVIDER`) selects the active
one. The SkyPilot provider is an optional extra, `pip install 'flow-compute[skypilot]'`, and
is not installed by default. Switching providers means switching configurations; a single
config does not span providers.

---

## Under the Hood (Advanced)

* **Bid Caps** – Protect budgets automatically.
* **Self-Healing** – Spot nodes dynamically migrate tasks.
* **Docker/Conda** – Pre-built images or dynamic install.
* **Multi-cloud Ready** – Mithril (with Oracle, Nebius integrations internal to Mithril), and more coming
* **SLURM Compatible** – Run `#SBATCH` scripts directly.

---

## Python SDK

### Advanced Task Configuration

```python
# Distributed training example (32 GPUs, Mithril groups for InfiniBand connectivity by default)
task = flow.run(
    "torchrun --nproc_per_node=8 train.py",
    instance_type="8xa100",
    num_instances=4,  # Total of 32 GPUs (4 nodes × 8 GPUs each)
    env={"NCCL_DEBUG": "INFO"}
)

# Mount S3 data + persistent volumes
task = flow.run(
    "python analyze.py",
    instance_type="a100",
    data_mounts={
        "/datasets": "s3://ml-bucket/imagenet",  # S3 via s3fs
        "/models": "volume://pretrained-models"   # Persistent storage
    }
)
```

The vocabulary is the same on every surface (`instance_type`, `env`, `data_mounts`, ...);
see [docs/vocabulary.md](docs/vocabulary.md).

### Key Features Summary

* **Distributed Training** – Multi-node InfiniBand clusters auto-configured
* **Code Upload** – Automatic with `.flowignore` (or `.gitignore` fallback)
* **Live Debugging** – SSH into running instances (`flow ssh`)
* **Cost Protection** – Built-in `max_price_per_hour` safeguards
* **Jupyter Integration** – Connect notebooks to GPU instances

**Documentation**: https://docs.mithril.ai/cli-and-sdk/quickstart

## Further Reading

* [Restoring the Promise of Public Cloud for AI](https://mithril.ai/blog/restoring-the-promise-of-the-public-cloud-for-ai)
* [Introducing Mithril](https://mithril.ai/blog/introducing-foundry)
* [Spot Auction Mechanics](https://docs.mithril.ai/compute-and-storage/spot-bids#spot-auction-mechanics)
