Metadata-Version: 2.5
Name: visionset
Version: 0.0.1b4
Summary: Open-source, local-first tool for creating, curating, and versioning computer-vision training datasets — with a Physical AI roadmap.
Project-URL: Homepage, https://github.com/Robomous/VisionSet
Project-URL: Issues, https://github.com/Robomous/VisionSet/issues
Author: Robomous Inc.
License-Expression: Apache-2.0
License-File: LICENSE
License-File: THIRD-PARTY-NOTICES.md
Keywords: annotation,computer-vision,datasets,labeling,physical-ai
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: Image Recognition
Requires-Python: >=3.11
Requires-Dist: fastapi>=0.141.1
Requires-Dist: mcp>=2.0.0
Requires-Dist: packaging>=24.0
Requires-Dist: pillow-heif>=1.5.0
Requires-Dist: pillow>=11.0
Requires-Dist: pydantic-settings>=2.4
Requires-Dist: pydantic>=2.7
Requires-Dist: python-multipart>=0.0.9
Requires-Dist: sqlalchemy>=2.0
Requires-Dist: typer>=0.12
Requires-Dist: uvicorn>=0.52.0
Provides-Extra: local-inference
Requires-Dist: accelerate>=0.33; extra == 'local-inference'
Requires-Dist: huggingface-hub>=0.24; extra == 'local-inference'
Requires-Dist: torch>=2.4; extra == 'local-inference'
Requires-Dist: torchvision>=0.19; extra == 'local-inference'
Requires-Dist: transformers>=5.0; extra == 'local-inference'
Description-Content-Type: text/markdown

<img src="https://cdn.robomous.ai/public-images/robomous-banner.svg" alt="Robomous.ai" width=300 />

-----

# Robomous VisionSet

[![CI](https://github.com/robomous/visionset/actions/workflows/ci.yml/badge.svg)](https://github.com/robomous/visionset/actions/workflows/ci.yml)
[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/visionset.svg)](https://pypi.org/project/visionset/)
[![License: Apache 2.0](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](LICENSE)

**VisionSet** is an open-source, local-first, SDK-first tool by [Robomous](https://robomous.ai)
for creating, curating, and versioning computer-vision training datasets. Today it targets 2D
image annotation; the domain model is built for a Physical AI roadmap — 3D point clouds, lane
labeling, and multimodal data land on the same foundations. Your data stays on your machines,
every surface (UI, CLI, MCP) is a thin client of the same SDK, and the release artifact is a
plain `pip` package.

## What it does

Point it at a folder of images or a clip of video, label them, and hand a trainer a dataset —
without a server, an account, or your pixels leaving the machine.

| | |
| --- | --- |
| **Ingest** | folders and video. Frames are cut, hashed and stored by content, so the same file twice is one asset and a re-run costs nothing. |
| **Annotate** | boxes, polygons and classification tags in the browser, with undo/redo, keyboard-first tools, and a headless engine underneath that the UI is only one renderer of. |
| **Version** | schema versions are immutable and every label records the one it was judged against. A release freezes the whole thing into a manifest; publish twice from unchanged data and the bytes are identical. |
| **Split** | a stored recipe rather than a materialised assignment, keyed on **content hash** — so two copies of one image cannot straddle a train/test boundary. |
| **Export** | addressed to [the model you will train](docs/content/releases.md#export-targets) — the Ultralytics YOLO line, YOLOv7, COCO, Pascal VOC, classification and [the lane family](docs/content/releases.md#the-lane-formats) — each format declaring what it can carry. VisionSet works out exactly what a target would drop *before* writing anything, and refuses to drop it silently. |
| **Pre-process** | a named [recipe](docs/content/preprocessing.md) applied at export: resize every image, and write augmented variants of the training images beside their sources. The release stays untouched, and the export report records the recipe it ran with. |
| **Auto-label** | a model you configure and fetch yourself, never one that arrives on its own. Click a point and SAM 2 proposes the shape under it; type words and Grounding DINO finds what they name. Every suggestion is a proposal until you accept it, and an accepted one records which model produced it. |
| **Automate** | one SDK under everything, reachable as a Python API, a REST API, a CLI, and an MCP server an agent can drive. |

## Quickstart

```bash
uv tool install "git+https://github.com/Robomous/VisionSet"   # PyPI lands with the beta

visionset init ~/datasets/road-signs     # a workspace, here and nowhere else
cd ~/datasets/road-signs
visionset server                         # API at http://127.0.0.1:8000, app at /app
```

Then follow [the tutorial](docs/content/tutorial.md): a clip of video to a YOLO dataset in about half an
hour. Full prerequisites — Python 3.12, and a browser, which is where a video is decoded — are in
[docs/content/install.md](docs/content/install.md).

`init` is the only command that creates a workspace, and it refuses a directory that already holds
something. `visionset server` run outside one refuses with one sentence and exit 1; it never creates
one, because a command that silently made a workspace out of whatever directory you were standing
in is how data ends up somewhere nobody chose.

Or hand the workspace to an agent — the same cycle, over
[MCP](https://modelcontextprotocol.io), with the tools an agent needs to *look* at what it is
labelling:

```json
{ "mcpServers": { "visionset": {
    "command": "visionset", "args": ["mcp"],
    "env": { "VISIONSET_WORKSPACE": "/path/to/workspace" } } } }
```

The whole cycle as tools, plus the four deletions that are offered only when the server is started
with `--allow-destructive` — because a `confirm` parameter is documented in the same listing an
agent reads before choosing, and four of four measured runs sent it on the first call. See
[docs/content/mcp.md](docs/content/mcp.md) for how a client is configured and why each tool exists,
[docs/content/mcp-tools.md](docs/content/mcp-tools.md) for the generated reference, or
[docs/content/mcp-walkthrough.md](docs/content/mcp-walkthrough.md) for a session start to finish — including what
twelve real agent runs actually did with it.

Or drive the whole cycle from the terminal, without a server:

```bash
visionset project create road-signs
visionset schema apply schema.json --project road-signs
BATCH=$(visionset ingest ./incoming --project road-signs)
visionset batch approve "$BATCH" --jobs-of 100 --start
# …annotate, then…
visionset batch complete "$BATCH" --promote
visionset release publish --tag v1.0 --project road-signs --split 0.7,0.15,0.15
visionset export --project road-signs --release v1.0 --target yolo11 --out ./out --allow-lossy
```

Every command takes `--json` for scripting, and the shapes are the REST API's. See
[docs/content/cli.md](docs/content/cli.md), or [`examples/cli_end_to_end.sh`](examples/cli_end_to_end.sh) for that
walk with its assertions still in it.

Prefer to see the SDK first? [`examples/sdk_end_to_end.py`](examples/sdk_end_to_end.py) drives an
empty directory to a hash-verified release in one pass, generating its own images — no server,
no CLI, nothing to download. Run it with `uv run python examples/sdk_end_to_end.py`; the
walkthrough is in [docs/content/examples.md](docs/content/examples.md).

For where the assets themselves come from,
[`examples/ingest_end_to_end.py`](examples/ingest_end_to_end.py) turns a folder of generated stills
into 50 deduplicated assets in an approved batch, then shows a re-run creating nothing. It needs
nothing on the host. Starting from a video is a browser capability and has no script form: see
[docs/content/ingest.md](docs/content/ingest.md).

The same cycle runs over each of the other two surfaces, and both start the shipped command for
real: [`examples/http_end_to_end.py`](examples/http_end_to_end.py) starts `visionset server` on a
free port and drives the API with `urllib` and a bearer token — multipart upload, 202-and-poll
ingest, hash-checked manifest and a 401 it asserts — while
[`examples/mcp_end_to_end.py`](examples/mcp_end_to_end.py) spawns `visionset mcp` and talks
JSON-RPC down its pipe, scaling every box out of the preview it saw and into the asset's own
pixels.

## Monorepo map

```
src/visionset/          Single Python distribution (one wheel, one import namespace)
  kernel/               Hexagonal core: domain + ports + default adapters (framework-free)
  wire/                 The JSON shapes the CLI and MCP publish (gated against the REST models)
  server/               FastAPI — exposes the SDK via REST; openapi.json is a committed contract
  cli/                  Typer CLI (`visionset` console script)
  mcp/                  MCP server (stdio) — 56 agent tools over the same SDK, four more on request
  formats/              Exporter plugins: ultralytics, yolov5-yaml, coco, voc, classification and
                        the five lane formats (entry-point group `visionset.formats`)
  preprocessing/        Pre-processing drivers: Pillow resize and augmentation behind the
                        `PreprocessingDriver` port (entry-point group `visionset.preprocessing`)
  jobs/                 Handlers for work that outlives a request: ingest, export, weights
  inference/            Where a model connection becomes a running model (optional runtime)
  _static/              Compiled UI bundle lands here at build time (ships in the wheel)
frontend/
  annotator/            @visionset/annotator — headless annotation engine (no React in core/)
  ui-core/              @visionset/ui-core — domain components, tokens, generated API contract
  app/                  @visionset/app — OSS product shell (Vite + React, never published)
tests/                  Python tests, incl. machine-enforced architecture contracts
examples/               Six runnable end-to-end scripts, all exercised in CI
docs/                   Documentation: content/ is the Markdown (user and contributor docs, one page per
                        subsystem); the rest is the Astro + Starlight site that renders it — a view, never a second copy
docker/                 Dev-only compose environment (never the release artifact)
scripts/                Repo automation (OpenAPI export, version sync, bundling, dist build)
.agents/skills/         Coding-agent skills, tool-agnostic (see AGENTS.md)
```

`ui-core` reaches data through a small contract rather than any particular transport, so its
screens are mountable by a host other than this application — the OSS app you see here is one
implementation of that host, not the only one there could be.

## Documentation

Start with [docs/content/install.md](docs/content/install.md) and [docs/content/tutorial.md](docs/content/tutorial.md).
[docs/content/README.md](docs/content/README.md) indexes the rest — one page per subsystem, each written to
explain the decisions rather than restate the code.

**`docs/content/` is the source of truth**, and it is plain Markdown so that it reads on GitHub with
nothing installed. [`docs/`](docs/README.md) renders the same files as a searchable
website; it adds no content of its own. To read it locally:

```bash
docker compose -f docker/compose.yaml up docs    # http://localhost:4321
```

or, without Docker, `pnpm --dir docs install && pnpm --dir docs dev`. Editing anything
under `docs/content/` reloads the page.

## Development setup

```bash
uv sync         # Python env + dev tools
pnpm install    # frontend workspace
```

Then `uv run visionset server` and `pnpm --filter @visionset/app dev`. Or run the whole thing in
containers instead, with nothing installed on the host and nothing built.

### Run it with Docker, and sign in with nothing

```bash
docker compose -f docker/compose.yaml up
```

**Open http://localhost:8080. There is no token to find and nothing to paste** — the app opens on
the project list. The server signs in the browser it served itself, over an `HttpOnly` cookie it
sets on the first request the page makes; [docs/content/auth.md](docs/content/auth.md#the-browser-session) has the
mechanism and the reasoning.

One port, nginx in front of both services. The first run builds two images, every later one just
starts them; dependencies are installed at build time, so starting the stack downloads nothing.

A token is still minted on first boot and printed in the `api` logs, because `curl`, the SDK and
MCP clients have no session and never will:

```bash
docker compose -f docker/compose.yaml logs api | grep vst_    # if you scrolled past it
docker compose -f docker/compose.yaml exec api \
  visionset token create --name <name>                        # or mint another
```

The browser never needs either. If the page *does* ask for a token, the stack is not the one this
README describes — check that `VISIONSET_UI_SESSION: always` is set on the `api` service and that
you are reaching it through port 8080.

> **Why `always` here, and what it costs.** The default, `VISIONSET_UI_SESSION=auto`, issues a
> session only to a client on this machine — and behind a proxy no request ever looks like one,
> because the peer is nginx. So the compose stack says `always`. The front door on 8080 is
> published on every interface, so another device on your network — a phone, a tablet — can open
> the dev stack at `http://<your address>:8080`; the api and vite publish no host port at all. The
> consequence is that whoever reaches 8080 is signed in, so the stack trusts the network it runs
> on. On one that is not yours, set `VISIONSET_UI_SESSION: never` to go back to typing a token,
> or bind the nginx port to `127.0.0.1` in `docker/compose.yaml`.

Everything it stores lands in **`workspace-data/`** (git-ignored): SQLite for metadata, a local
directory for the files, one workspace holding both — the shape MLflow's default mode has, and the
only shape VisionSet has. Put it elsewhere with `VISIONSET_DATA=/path docker compose …`; it is a
bind mount, so `down -v` does not take your data with it.

The containers run as you rather than as root, so everything they write there — and every
`__pycache__` and generated page elsewhere in the checkout — belongs to the account that started
them, and a later `pnpm -r build` or `git worktree remove` is not blocked by a file it cannot
touch. That is the uid the stack finds at 1000, which is the first account on a Linux
workstation. If yours is another number, tell it once and rebuild:

```bash
printf 'VISIONSET_UID=%s\nVISIONSET_GID=%s\n' "$(id -u)" "$(id -g)" > docker/.env
docker compose -f docker/compose.yaml up --build
```

On macOS and Windows the defaults are already right — Docker Desktop translates ownership itself.
Under rootless Docker, set both to `0`: there the daemon maps the container's root onto you
already.

Dev only — the release artifact is always the pip package.

The inner loop while iterating: `uv run pytest <path>`, `uv run lint-imports`,
`uv run mypy src/visionset/kernel` (the kernel only — CI checks all of `src/visionset`),
`pnpm -r build`, `pnpm test` (no browser — the two Playwright suites sit outside it). Each is a
subset of what CI runs on every pull request; the full list, with what each command does not
cover, is in [CONTRIBUTING.md](CONTRIBUTING.md#checks-that-must-stay-green).

## Releases

[CHANGELOG.md](CHANGELOG.md) — what each version added, and the six milestones that got here.
[docs/content/releasing.md](docs/content/releasing.md) is the runbook for cutting one.

## License

Apache-2.0 — copyright Robomous Inc. See [LICENSE](LICENSE).

The wheel ships a compiled browser bundle, so it redistributes its dependencies too;
[THIRD-PARTY-NOTICES.md](THIRD-PARTY-NOTICES.md) records the notices that owes.
