Metadata-Version: 2.5
Name: fused
Version: 2.9.3b10
Summary: CLI and platform for end-to-end data work with cloud-native datasets
License: MIT
License-File: LICENSE
Requires-Python: >=3.11
Requires-Dist: aiohttp<4,>=3.8.3
Requires-Dist: click>=8.1.7
Requires-Dist: fastapi>=0.100.0
Requires-Dist: fsspec>=2024.3.1
Requires-Dist: httpx>=0.27
Requires-Dist: keyring>=24
Requires-Dist: loguru<0.8,>=0.7.0
Requires-Dist: packaging>=23
Requires-Dist: pluggy>=1.5
Requires-Dist: pydantic<3,>=2.7.0
Requires-Dist: pyjwt>=2.10
Requires-Dist: pyyaml>=6.0
Requires-Dist: requests<3,>=2.31
Requires-Dist: rtoml<1,>=0; sys_platform != 'emscripten'
Requires-Dist: tabulate>=0.10.0
Requires-Dist: tomlkit>=0.13
Requires-Dist: tqdm<5,>=4.67.1
Requires-Dist: uvicorn>=0.23.0
Requires-Dist: yarl<2,>=1.9.4
Provides-Extra: all
Requires-Dist: anthropic>=0.40.0; extra == 'all'
Requires-Dist: boto3>=1.34.0; extra == 'all'
Requires-Dist: dropbox; extra == 'all'
Requires-Dist: fastmcp>=3.2.4; extra == 'all'
Requires-Dist: geopandas<2,>=0.14; extra == 'all'
Requires-Dist: hubspot-api-client; extra == 'all'
Requires-Dist: importlib-resources<6,>=5.12.0; extra == 'all'
Requires-Dist: mcp<2,>=1.8.0; extra == 'all'
Requires-Dist: mercantile<2,>=1.2.1; extra == 'all'
Requires-Dist: numpy>=1.20; extra == 'all'
Requires-Dist: pandas<3,>=2.2; extra == 'all'
Requires-Dist: pillow>=10; extra == 'all'
Requires-Dist: pyarrow>=14; extra == 'all'
Requires-Dist: pyjwt[crypto]>=2.10; extra == 'all'
Requires-Dist: pyproj<4,>=3.7.1; extra == 'all'
Requires-Dist: rasterio<2,>=1.3.10; extra == 'all'
Requires-Dist: rioxarray<0.16,>=0.15; extra == 'all'
Requires-Dist: selenium>=4.44.0; extra == 'all'
Requires-Dist: shapely<3,>=2; extra == 'all'
Requires-Dist: snowflake-connector-python>=3.0.0; extra == 'all'
Requires-Dist: ty>=0.0.1a1; extra == 'all'
Requires-Dist: xarray>=2024.3; extra == 'all'
Provides-Extra: aws
Requires-Dist: boto3>=1.34.0; extra == 'aws'
Requires-Dist: numpy>=1.20; extra == 'aws'
Requires-Dist: pandas<3,>=2.2; extra == 'aws'
Requires-Dist: pyarrow>=14; extra == 'aws'
Requires-Dist: pyjwt[crypto]>=2.10; extra == 'aws'
Provides-Extra: batch
Requires-Dist: geopandas<2,>=0.14; extra == 'batch'
Requires-Dist: importlib-resources<6,>=5.12.0; extra == 'batch'
Requires-Dist: mercantile<2,>=1.2.1; extra == 'batch'
Requires-Dist: numpy>=1.20; extra == 'batch'
Requires-Dist: pandas<3,>=2.2; extra == 'batch'
Requires-Dist: pillow>=10; extra == 'batch'
Requires-Dist: pyarrow>=14; extra == 'batch'
Requires-Dist: pyproj<4,>=3.7.1; extra == 'batch'
Requires-Dist: rasterio<2,>=1.3.10; extra == 'batch'
Requires-Dist: rioxarray<0.16,>=0.15; extra == 'batch'
Requires-Dist: shapely<3,>=2; extra == 'batch'
Requires-Dist: xarray>=2024.3; extra == 'batch'
Provides-Extra: data
Requires-Dist: numpy>=1.20; extra == 'data'
Requires-Dist: pandas<3,>=2.2; extra == 'data'
Requires-Dist: pyarrow>=14; extra == 'data'
Provides-Extra: dropbox
Requires-Dist: dropbox; extra == 'dropbox'
Provides-Extra: geo
Requires-Dist: geopandas<2,>=0.14; extra == 'geo'
Requires-Dist: importlib-resources<6,>=5.12.0; extra == 'geo'
Requires-Dist: mercantile<2,>=1.2.1; extra == 'geo'
Requires-Dist: numpy>=1.20; extra == 'geo'
Requires-Dist: pandas<3,>=2.2; extra == 'geo'
Requires-Dist: pillow>=10; extra == 'geo'
Requires-Dist: pyarrow>=14; extra == 'geo'
Requires-Dist: pyproj<4,>=3.7.1; extra == 'geo'
Requires-Dist: rasterio<2,>=1.3.10; extra == 'geo'
Requires-Dist: rioxarray<0.16,>=0.15; extra == 'geo'
Requires-Dist: shapely<3,>=2; extra == 'geo'
Requires-Dist: xarray>=2024.3; extra == 'geo'
Provides-Extra: hubspot
Requires-Dist: hubspot-api-client; extra == 'hubspot'
Provides-Extra: local
Requires-Dist: keyrings-alt>=5; extra == 'local'
Provides-Extra: mcp
Requires-Dist: mcp<2,>=1.8.0; extra == 'mcp'
Provides-Extra: raster
Requires-Dist: geopandas<2,>=0.14; extra == 'raster'
Requires-Dist: importlib-resources<6,>=5.12.0; extra == 'raster'
Requires-Dist: mercantile<2,>=1.2.1; extra == 'raster'
Requires-Dist: numpy>=1.20; extra == 'raster'
Requires-Dist: pandas<3,>=2.2; extra == 'raster'
Requires-Dist: pillow>=10; extra == 'raster'
Requires-Dist: pyarrow>=14; extra == 'raster'
Requires-Dist: pyproj<4,>=3.7.1; extra == 'raster'
Requires-Dist: rasterio<2,>=1.3.10; extra == 'raster'
Requires-Dist: rioxarray<0.16,>=0.15; extra == 'raster'
Requires-Dist: shapely<3,>=2; extra == 'raster'
Requires-Dist: xarray>=2024.3; extra == 'raster'
Provides-Extra: snowflake
Requires-Dist: snowflake-connector-python>=3.0.0; extra == 'snowflake'
Provides-Extra: vector
Requires-Dist: geopandas<2,>=0.14; extra == 'vector'
Requires-Dist: importlib-resources<6,>=5.12.0; extra == 'vector'
Requires-Dist: mercantile<2,>=1.2.1; extra == 'vector'
Requires-Dist: numpy>=1.20; extra == 'vector'
Requires-Dist: pandas<3,>=2.2; extra == 'vector'
Requires-Dist: pillow>=10; extra == 'vector'
Requires-Dist: pyarrow>=14; extra == 'vector'
Requires-Dist: pyproj<4,>=3.7.1; extra == 'vector'
Requires-Dist: rasterio<2,>=1.3.10; extra == 'vector'
Requires-Dist: rioxarray<0.16,>=0.15; extra == 'vector'
Requires-Dist: shapely<3,>=2; extra == 'vector'
Requires-Dist: xarray>=2024.3; extra == 'vector'
Provides-Extra: verify
Requires-Dist: anthropic>=0.40.0; extra == 'verify'
Requires-Dist: ty>=0.0.1a1; extra == 'verify'
Provides-Extra: workbench
Requires-Dist: fastmcp>=3.2.4; extra == 'workbench'
Requires-Dist: selenium>=4.44.0; extra == 'workbench'
Description-Content-Type: text/markdown

# fused

[![CI](https://github.com/fusedio/fused/actions/workflows/ci.yml/badge.svg)](https://github.com/fusedio/fused/actions/workflows/ci.yml)

Fused agent toolkit — the data SDK is `fused`; the agent commands are top-level, the legacy proprietary SDK CLI is under `fused workbench`.

A CLI for end-to-end data work with cloud-native datasets.

**Find → Load → Explore → Analyze → Decide → Act**

It gives AI agents and developers a single command set to explore, run code against, and act on cloud data — with the compute environment managed for you.

Data lives in S3 as Parquet/Arrow files. Code runs in AWS Lambda — or fully locally when you don't have (or don't want) a cloud account, or before one is provisioned. Agents drive it by running the same `fused` commands a person would.

### The surface

fused is the **core layer that executes data work** — the platform that data agents and apps build on. Its surface is the **`fused` CLI** (`fused …` / `uv run fused …`); running `fused` with no subcommand prints the help. (The former root MCP server and the bare widget UI — `fused widget …` / `fused dev serve` — have been removed.)

One command still speaks MCP: `fused app serve` exposes a single app folder's `mcp.toml` tools as a stdio MCP server, which [fused-render](https://github.com/fusedio/fused-render) registers with Claude. See [`spec/serve/app-mcp.md`](spec/serve/app-mcp.md).

The full UI-first app experience — the local web control plane for the complete agent-orchestration workflow — is a separate product, **[Flow](https://github.com/fusedio/flow)** (`fusedio/flow`), that consumes fused's CLI.

## Capabilities

fused gives an agent (or a person at the CLI) the tools for an end-to-end data workflow:

- **Find & inspect data** — list buckets and objects, count and filter files, read a Parquet/Arrow/CSV schema and row count, and mint presigned download URLs.
- **Run code against that data** — execute Python in an isolated, per-call sandbox with the environment's packages and input files, returning a `result`; run pytest suites with line and branch coverage.
- **Scale out** — fan out across partitions from inside an execution using the same coordinator/worker pattern on every backend.
- **Verify & audit** — scan code, dependencies, and input files before running them (`fused code verify`: static and dependency scans, an optional LLM spec check), and read the audit log (`fused audit log`).
- **Cache** — content-address results so a repeat call returns a stored value instead of re-executing.
- **Serve & share** — expose a route's code as an HTTP endpoint, locally or as a managed cloud deployment; publish apps and [fused-render](https://github.com/fusedio/fused-render) page bundles as opaque public share links (`fused share create`) on the managed backend or a provisioned AWS serving plane. A published page can optionally be made cloneable (`--allow-clone`), so a viewer who can reach it may download its source bundle from the page's own URL and take it back to a local folder.
- **Manage secrets & infrastructure** — read/write provider secrets and plan/apply/teardown the cloud resources each environment needs.

The same commands work across [pluggable backends](#backends), so code written against fused doesn't change when the execution target does. See [Commands](#commands) for the full list, or [`spec/overview.md`](spec/overview.md) for a deeper tour.

## Quickstart

```bash
uv sync
uv run fused --help
```

### Verify it works

Create an environment and run a one-line smoke test. **You don't need a cloud account to get started:** the local backend runs code on your machine in a cached virtualenv.

```bash
# Option A — AWS: runs code in Lambda, auto-provisioning a managed IAM role +
# S3 cache bucket; the compute function runs the env's container image
# (build it with `infra build-image`)
uv run fused env create prod --backend aws

# Option B — local, no AWS account required: runs code on this machine in a
# cached virtualenv (created with uv when available, else pip)
uv run fused env create dev --backend local

# Run code — with a single env, it is selected automatically
uv run fused code run -c "result = 1 + 1"   # -> result: 2
```

The paths are independent: the local backend makes no AWS calls, and AWS needs only credentials. For a guided first-time setup (backend choice, AWS credential checks, provisioning, and teardown), see the bundled `agent-core:fused-setup` skill.

On first use against AWS, fused automatically creates and manages the IAM role needed to run Lambda functions. You can override this with `OPENFUSED_ROLE_ARN` if you prefer to supply your own role.

Commands that permanently delete resources (`secrets delete`, `env delete`, `infra teardown`, `udf retire`) ask for confirmation unless you pass `--yes`.

## Backends

Fused runs the same tools against a pluggable backend. Two run on infrastructure you control, plus one Fused-hosted backend:

- **AWS** (`aws`, default) — storage in S3, compute in AWS Lambda, secrets in AWS Secrets Manager. The path for production and scale: it fans out horizontally and needs only AWS credentials (fused creates and manages the IAM role and supporting resources for you). The compute function runs the environment's container image, built and pushed with `infra build-image`.
- **Local** (`local`) — storage on the local filesystem, compute on the host in a cached virtualenv built with uv or pip, secrets in an encrypted local file. No cloud account required. Use it for development, offline/air-gapped work, CI, or before a cloud environment is provisioned; code runs directly on the host, so there is no container isolation.
- **Fused** (`fused`) — Fused's hosted, managed environment, reached over its data-plane endpoint (fused acts as an MCP client of the hosted tool surface). Code runs in a per-tenant sandbox Fused operates; the local side provisions nothing. See [`spec/onboarding/fused-onboarding.md`](spec/onboarding/fused-onboarding.md) for login → key → env setup.

All implement the same contract, so code written against one runs unchanged on the others — switching is just a config change. Select the backend with `OPENFUSED_BACKEND=aws|local` (see [docs/configuration.md](docs/configuration.md)) or per named environment with `fused env create --backend …`.

## Commands

Run `fused <group> --help` for the options of each command; the design reference is [`spec/cli/cli.md`](spec/cli/cli.md).

| Command group | What it does |
|---|---|
| `fused files list\|count\|get\|schema\|upload` | List buckets and objects, count files (optionally by extension), mint a presigned download URL, read a Parquet/Arrow IPC/CSV schema and row count, upload content |
| `fused code run` | Run Python on the resolved environment; assign `result` to return a value, register a `@fused.udf`, or pass `--entrypoint NAME` to call that function directly. Ship files with `--input-file` (repeatable; each lands under its basename). On the local backend, `--project` / `--project-dir` selects the project's `.venv` |
| `fused code test` | Run a pytest file against user code; reports per-test outcomes plus line and branch coverage. On the local backend `--project` or `--project-dir` is required (pytest/coverage must be dev deps) |
| `fused code verify` | Scan code, dependencies, and input files for security issues without executing; exits 1 on any BLOCK finding |
| `fused audit log` | Recent security audit events from the local SQLite store (`~/.openfused/audit.db`, optionally merged with S3); filter by event type, status, or project |
| `fused secrets get\|put\|list\|delete` | Read, write, list, and delete provider secrets (AWS Secrets Manager, or the encrypted local store). `delete` prompts unless `--yes` |
| `fused project new\|list\|show\|status\|…` | Scaffold and list workspace projects; `show` prints the project's context packet (identity, SKILL.md contract, dataset notes, UDF scripts, resolved environment); `status` shows live deploy status from the resolved env's cloud snapshot |
| `fused env create\|update\|delete\|list\|show\|default` | Manage named environments. `delete` removes the config only (cloud resources untouched) and prompts unless `--yes` |
| `fused infra plan\|apply\|build-image\|lambda-reset\|teardown\|serve` | Plan and reconcile an environment's cloud resources, build its container image, reset its Lambda functions, or tear everything down (`teardown` prompts unless `--yes`) |
| `fused udf deploy\|promote\|rollback\|retire` | Deploy a UDF to its preview or release channel, promote preview to release, roll release back, or retire it (`retire` prompts unless `--yes`) |
| `fused share …` | Publish apps and mint/manage served URLs; `share cache-clear TOKEN` busts a deployed route's cached results |
| `fused app serve` | Serve one app folder's `mcp.toml` tools as a stdio MCP server ([`spec/serve/app-mcp.md`](spec/serve/app-mcp.md)) |
| `fused doctor` | Survey every project in every workspace for health issues; `--fix` remediates |

> **Consuming-app teamwork tool (Tier B, not part of fused).** A consuming app such as
> [Flow](https://github.com/fusedio/flow) injects its own loopback teamwork MCP into the agent runs it
> spawns — e.g. an approval gate requested through `ask_user(… effect: "approval_gate", effectArgs: { verb, detail })`
> (`verb`: `storage_write` \| `secret_write` \| `external_side_effect`) that an agent calls **before** a gated
> Act to request human approval (the approval-gate contract lives in `spec/security/autonomy.md` §3;
> cooperative/advisory). That surface is the consuming app's, not fused's.

## Configuration

Configure fused with [named environments](#named-environments) (`fused env`) or environment variables. The full list of variables and their defaults lives in [docs/configuration.md](docs/configuration.md).

## Named environments

`fused env` manages named configurations stored in `~/.openfused/envs.json`, so you can switch between backends without managing environment variables manually:

```bash
# Create and provision a named AWS environment
fused env create prod --backend aws --prefix myapp-

# Run a command against a specific environment
fused --env prod secrets list
fused --env staging secrets list

# Pin a project's default environment (validated against stored envs)
fused project set my-project --env prod
```

Environment resolution order for every command:
1. `--env` flag or `OPENFUSED_ENV` env var → explicit override
2. Inside a project with `[project].default_env` set in `openfused.toml` → manifest pin
3. Exactly one environment exists → selected automatically (sole-env auto)
4. Multiple environments, no pin → error: set `default_env` or pass `--env`

See `fused env --help` for the full subcommand list.

## Observability

Logs, CloudWatch metrics (Lambda + cache), the audit log, and the system signals
an operator should watch (concurrency, cache hit rate, bucket/DB growth) are
documented in [docs/observability.md](docs/observability.md).

## Examples

Runnable examples live in [`examples/`](examples/) — from [`hello_world_python.py`](examples/hello_world_python.py) and single-file DuckDB queries to multi-step pipelines (`examples/github_archive/`, `examples/ais_sf_ferries/`) and Lambda fan-out across partitions. Run any of them with:

```bash
uv run fused code run examples/hello_world_python.py --file
```

For the conventions these examples follow when you build your own project, see [docs/project-structure.md](docs/project-structure.md).

## Requirements

- Python 3.11+ and [uv](https://docs.astral.sh/uv/)
- For the AWS backend: AWS credentials with permissions to create Lambda functions, manage IAM roles, and read/write S3. The environment's container image is built in **AWS CodeBuild by default** (no local Docker); pass `--builder local` to build with a host Docker daemon instead
- For the local backend: nothing beyond Python — uv is used for fast venv creation when present, else pip

### Optional extras

`pip install fused` alone gives you the CLI, the **local** backend and `import
fused`. It does not install `mcp`, boto3, `cryptography`, `ty`, `anthropic`,
pyarrow, pandas or numpy. Everything heavier is an opt-in extra:

| Extra | Installs | Needed for |
|---|---|---|
| `data` | pyarrow, pandas, numpy | `fused.run()` / `fused.submit()` on DataFrame results, `get_file_schema`, the SDK's schema conversions and table/H3 readers |
| `mcp` | mcp (`>=1.8.0,<2`) | `fused app serve` (the per-app stdio MCP server fused-render registers with Claude) and the **fused** backend, whose data plane is an MCP endpoint |
| `aws` | boto3, `pyjwt[crypto]` (i.e. `cryptography`) + `data` | the **aws** backend and everything built on it: `fused infra plan/apply/serve/lambda-reset`, the serve plane and its JWS verification, `fused share`'s AWS paths, the audit log's S3 sink |
| `verify` | ty, anthropic | the verify pipeline's type-checker scanner and AI-judged spec scanner |
| `geo` | shapely, geopandas, pyproj, mercantile, pillow, xarray, rioxarray, rasterio + `data` | the vendored SDK's raster/vector/tile helpers |
| `batch`, `raster`, `vector` | same as `geo` | legacy SDK names, kept as aliases |
| `workbench` | selenium, fastmcp | the `fused workbench` CLI's `json-ui run` and MCP/canvas commands |
| `local` | keyrings.alt | a file-based local secrets store on hosts with no OS keychain (plaintext on disk) |
| `hubspot`, `snowflake`, `dropbox` | the matching client library | the SDK's connectors for those services |
| `all` | every extra above except `local` | the full surface, without silently enabling plaintext secrets |

A missing extra fails with an actionable message such as `pip install
'fused[data]'` at the point of use, not a bare `ModuleNotFoundError` or an import
error at startup. The verify scanners degrade to a WARN finding instead. The
widget resolver's compute sandbox installs its own requirements and does not
depend on these extras.

Install what you need, e.g. `pip install 'fused[mcp]'` for `fused app serve`,
`pip install 'fused[aws,verify]'`, or `pip install
'fused[all]'` for everything.

## Documentation

- [CONTRIBUTING.md](CONTRIBUTING.md) — development setup and architecture overview.
- [docs/building.md](docs/building.md) — how fused is built and distributed as a single pure-Python package, builder requirements, and the publish steps.
- [docs/project-structure.md](docs/project-structure.md) — how to lay out a project built on fused (scripts, specs, workers/coordinators, tests).
- [docs/migrating-from-fused-py.md](docs/migrating-from-fused-py.md) — migrating from the hosted `fused` Python SDK (fused-py); see also [docs/fused-sdk-compatibility.md](docs/fused-sdk-compatibility.md) for the full name-by-name comparison.
- [`spec/`](spec/) — the design specification. Start with [`spec/overview.md`](spec/overview.md) for capabilities, entrypoints, and system parts; see [`spec/assumptions.md`](spec/assumptions.md) for runtime prerequisites. Each topic has a provider-neutral hub (`backends.md`, `environments.md`, `infra.md`) plus per-provider files (e.g. `-aws.md`, `-local.md`).
