Metadata-Version: 2.4
Name: adaptive-serve
Version: 0.1.0
Summary: Adaptive cloud-to-robot serving layer that scales robotics model quality to fit device, power, latency, and bandwidth constraints.
Author: Mandar Wagh
License: MIT
Project-URL: Homepage, https://github.com/mandarwagh9/adaptive-serve
Project-URL: Repository, https://github.com/mandarwagh9/adaptive-serve
Keywords: robotics,serving,inference,edge,vla,adaptive,cloud
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: dev
Requires-Dist: pytest>=8; extra == "dev"
Requires-Dist: ruff>=0.11; extra == "dev"
Provides-Extra: torch
Requires-Dist: torch>=2.0; extra == "torch"
Dynamic: license-file

<p align="center">
  <img src="https://raw.githubusercontent.com/mandarwagh9/adaptive-serve/main/assets/banner.png" alt="adaptive-serve" width="880">
</p>

<p align="center">
  <a href="https://github.com/mandarwagh9/adaptive-serve/actions/workflows/ci.yml"><img src="https://github.com/mandarwagh9/adaptive-serve/actions/workflows/ci.yml/badge.svg" alt="CI"></a>
  <img src="https://img.shields.io/badge/python-3.10+-EDE8DD?style=flat-square&labelColor=0B0E14" alt="python 3.10+">
  <img src="https://img.shields.io/badge/license-MIT-EDE8DD?style=flat-square&labelColor=0B0E14" alt="MIT">
  <img src="https://img.shields.io/badge/lint-ruff-FFB000?style=flat-square&labelColor=0B0E14" alt="ruff">
  <img src="https://img.shields.io/badge/tests-152_passing-FFB000?style=flat-square&labelColor=0B0E14" alt="152 tests passing">
</p>

# adaptive-serve

Adaptive cloud-to-robot serving layer for robotics models of all types.

It takes any trained robotics model and serves it to a robot, scaling model
quality up or down to fit the binding constraint: on-robot compute (TOPS/VRAM),
power budget, latency tolerance, and live network bandwidth. Think adaptive
bitrate streaming, but the renditions are model variants and the network is the
robot's compute and link.

The whole core runs with no GPU, no network, and no external dependencies. An
optional torch backend and a stdlib HTTP cloud adapter add real execution when
you want it.

## Why

A robot cannot round-trip to the cloud for every control step, and it must keep
acting if the link drops. adaptive-serve picks the right variant and serving mode
for the device in front of it, runs a slow cloud planner and a fast local reflex
loop at the same time, and degrades safely when the cloud goes away.

## Install

```bash
pip install -e ".[dev]"        # core plus test tooling
pip install -e ".[dev,torch]"  # also pull torch for the real backend
```

## Try it

```bash
adaptive-serve demo                                  # full end to end demo
adaptive-serve serve --model planner --vram 2 --bandwidth 200 --difficulty hard
adaptive-serve serve --model planner --vram 80 --bandwidth 0  --difficulty hard
```

The same model adapts to the device, exactly like adaptive bitrate:

```
device               mode    variant                 ~latency ms  meets
tiny-edge-offline    local   planner-vlm:2B:int8            36.0  False
edge-weak-link       split   planner-vlm:7B:fp16           761.8   True
edge-fast-link       cloud   planner-vlm:7B:fp16            21.0   True
workstation          local   planner-vlm:7B:fp16             4.2   True
```

And it keeps acting through a cloud outage, degrading from NOMINAL to a latched
emergency stop only when the last plan is long gone:

```
Multi-rate session with a cloud drop at 1000 ms:
  reflex steps: 1200
  plans published: 2
  safety mode timeline: {'nominal': 200, 'degraded': 531, 'hold': 400, 'kill': 69}
  final safety mode: kill
```

## Calibration: decide on real numbers, not guesses

By default the router uses portable heuristics for footprint and latency. To
ground its decisions in real hardware, measure actual models and route on those
numbers instead:

```bash
adaptive-serve calibrate            # measure real models -> calibration.json
adaptive-serve pipeline             # calibrate -> route on measured -> run in torch
```

`calibrate` records the exact parameter count, the weight memory at each
precision, and the measured forward latency per variant. A `ModelRegistry`
built with that table feeds the router measured footprints (heuristics remain the
fallback for any variant you have not measured), so a model that is actually small
gets served where the heuristic would have offloaded it.

Honest scope: weight bytes and params are device-independent and drive the VRAM
fit directly; latency is device-specific, so it stays informational. On a CPU box
the harness measures real but small reference models and INT8 via torch dynamic
quantization; the 7B/70B rungs and true INT4 need target hardware, which the same
harness supports. Requires the `torch` extra: `pip install -e ".[dev,torch]"`.

## Supported model classes

The registry and router are generic across these, never hardcoded to one:

- VLA / action policies (diffusion policies, ACT, flow-matching)
- world / dynamics models (Dreamer-style, latent predictors, video predictors)
- perception (detection, segmentation, depth, optical flow, 6DoF pose, keypoints)
- VLMs / high-level reasoners (planning, language grounding)
- state estimation / SLAM / odometry
- classical or learned planners and controllers

Each declares its role in the stack, native rate (Hz), latency budget, and which
scaling techniques it supports. Adding a new class touches no core.

## Architecture

```
DeviceProfile  ---+
                  |
ModelSpec --> Registry: variant ladder (size x quant) per model, any class
                  |
                  v
             Router: pick variant + quant + serving mode (local / cloud / split)
                  |
   +--------------+--------------+
   |                             |
Scaling strategies          Orchestrator: cloud System-2 planner ~1 Hz
size/quant/input/exit/      + local System-1 reflex 50-1000 Hz
cascade/split                    |
   |                             v
   +--------------->  Safety governor: hold / kill on loss
                             |
                             v
                      Runtime + backends: run local (stub or torch) or
                      offload over HTTP, fall back to local on failure
                             |
                             v
                      Telemetry: record every step
```

Two invariants hold throughout, both covered by tests:

- **Generic across model classes.** The router and scaling cores dispatch on a
  spec's declared role, tier, latency budget, and supported techniques, never on
  class identity.
- **The robot never blocks on the cloud.** The reflex tier runs every control
  tick. On cloud loss the planner stops publishing, the reflex keeps acting on
  its last plan, the safety governor degrades to HOLD and latches KILL when the
  plan is too stale, and a late or failed cloud call is dropped for a safe local
  action rather than awaited.

## Develop

```bash
ruff check .
pytest
```

See `STATUS.md` for the full component list and `PLAN.md` / `ADAPTERS_PLAN.md`
for the build history. MIT licensed.
