Metadata-Version: 2.5
Name: nodetool-core
Version: 0.8.2
Summary: Nodetool Core is the core library for Nodetool, providing the necessary functionality for building and running AI workflows.
Project-URL: Homepage, https://nodetool.ai
Project-URL: Repository, https://github.com/nodetool-ai/nodetool-core
Author-email: Matthias Georgi <matti.georgi@gmail.com>
License-File: LICENSE
Requires-Python: >=3.11
Requires-Dist: aiofiles>=25.1.0
Requires-Dist: aiohttp>=3.14.4
Requires-Dist: audioop-lts>=0.2.2; python_version >= '3.13'
Requires-Dist: brotli>=1.2.0
Requires-Dist: hf-transfer>=0.1.9
Requires-Dist: hf-xet>=1.7.0
Requires-Dist: httpx<1,>=0.28.1
Requires-Dist: huggingface-hub<2,>=1.33.0
Requires-Dist: msgpack>=1.2.3
Requires-Dist: numpy>=2.3.5
Requires-Dist: pillow<13,>=12.3.0
Requires-Dist: psutil>=7.2.2
Requires-Dist: pycares<5,>=4.11.0
Requires-Dist: pydantic>=2.14.0
Requires-Dist: pydub>=0.25.1
Requires-Dist: python-dotenv>=1.2.4
Requires-Dist: pyyaml>=6.0.2
Requires-Dist: safetensors>=0.8.0
Requires-Dist: tqdm>=4.70.1
Requires-Dist: websockets>=17.2
Provides-Extra: audio
Requires-Dist: pydub>=0.25.1; extra == 'audio'
Provides-Extra: data
Requires-Dist: ftfy>=6.3.1; extra == 'data'
Requires-Dist: joblib>=1.6.0; extra == 'data'
Requires-Dist: networkx>=3.6.1; extra == 'data'
Requires-Dist: pandas>=3.0.6; extra == 'data'
Requires-Dist: tiktoken>=0.14.0; extra == 'data'
Provides-Extra: dev
Requires-Dist: aioresponses>=0.7.9; extra == 'dev'
Requires-Dist: jsonschema>=4.26.0; extra == 'dev'
Requires-Dist: mypy>=2.4.0; extra == 'dev'
Requires-Dist: pytest-asyncio>=1.4.0; extra == 'dev'
Requires-Dist: pytest-cov>=7.1.0; extra == 'dev'
Requires-Dist: pytest-timeout>=2.4.0; extra == 'dev'
Requires-Dist: pytest-xdist>=3.8.0; extra == 'dev'
Requires-Dist: pytest>=9.1.1; extra == 'dev'
Requires-Dist: ruff==0.16.10; extra == 'dev'
Requires-Dist: ty>=0.0.85; extra == 'dev'
Provides-Extra: documents
Requires-Dist: html2text>=2025.4.15; extra == 'documents'
Requires-Dist: jinja2>=3.1.6; extra == 'documents'
Requires-Dist: markitdown>=0.1.8; extra == 'documents'
Requires-Dist: openpyxl>=3.1.5; extra == 'documents'
Requires-Dist: pdfplumber>=0.11.10; extra == 'documents'
Requires-Dist: pymupdf4llm>=1.28.2; extra == 'documents'
Requires-Dist: pymupdf>=1.28.2; extra == 'documents'
Requires-Dist: pypandoc>=1.17; extra == 'documents'
Requires-Dist: python-docx>=1.2.0; extra == 'documents'
Provides-Extra: ocr
Requires-Dist: paddleocr>=3.7.0; extra == 'ocr'
Requires-Dist: paddlepaddle>=3.3.1; extra == 'ocr'
Requires-Dist: pytesseract>=0.3.13; extra == 'ocr'
Provides-Extra: video
Requires-Dist: ffmpeg-python>=0.2.0; extra == 'video'
Requires-Dist: imageio-ffmpeg>=0.6.0; extra == 'video'
Requires-Dist: imageio>=2.38.1; extra == 'video'
Requires-Dist: opencv-python-headless>=5.0.0.93; extra == 'video'
Provides-Extra: viz
Requires-Dist: matplotlib>=3.11.2; extra == 'viz'
Requires-Dist: seaborn>=0.13.2; extra == 'viz'
Description-Content-Type: text/markdown

# NodeTool Core

<h3>Python Node System & Worker</h3>

<p align="center">
  <img src="https://img.shields.io/badge/Python-3.11%2B-blue.svg" alt="Python Version Badge">
  <img src="https://img.shields.io/badge/License-AGPL%20v3-blue.svg" alt="License Badge">
</p>

NodeTool Core is the Python library that provides the node system and worker subprocess for [NodeTool](https://github.com/nodetool-ai/nodetool). The TypeScript server handles HTTP API, workflow orchestration, agents, and chat. Python handles node execution and local-compute providers (HuggingFace, MLX).

______________________________________________________________________

## What's Here

- **Node system** — `BaseNode`, `ProcessingContext`, type metadata
- **Worker subprocess** — `python -m nodetool.worker` communicates with the TS server via WebSocket+MessagePack
- **Provider infrastructure** — Base classes and registry for local-compute providers
- **Media processing** — Image, audio, video conversion utilities
- **Storage** — Abstract storage backends (memory, file, S3)

## Quick Start

Install into a conda env named `nodetool`. The NodeTool server looks for a
Python interpreter in this order: `NODETOOL_PYTHON`, the active conda env when
it is named `nodetool`, then the usual `envs/nodetool` locations and the
desktop app's managed env. It does not find a project `.venv`.

```bash
conda create -n nodetool python=3.11 pandoc ffmpeg -c conda-forge
conda activate nodetool
uv pip install -e ".[dev]"   # or: pip install -e ".[dev]"

# Run tests
pytest -q

# Start a worker by hand (the TS server normally spawns it)
python -m nodetool.worker
```

Install node packs into the same env, for example
`uv pip install nodetool-huggingface`. To use a different interpreter, start
the server with `NODETOOL_PYTHON=/path/to/python`.

## Writing Nodes

```python
from nodetool.workflows.base_node import BaseNode
from nodetool.workflows.processing_context import ProcessingContext

class MyNode(BaseNode):
    """
    Brief description
    tags, keywords, for, search
    """
    input_text: str = ""

    async def process(self, context: ProcessingContext) -> str:
        return self.input_text.upper()
```

Nodes use `ProcessingContext` for media conversion (`image_to_pil`, `audio_from_numpy`, etc.), secrets (`get_secret`), asset storage, and progress reporting.

## Architecture

```
TS Server (Fastify)
    ├── HTTP API, WebSocket, Auth
    ├── Workflow orchestration (DAG scheduling)
    ├── Cloud providers (OpenAI, Anthropic, Gemini, ...)
    └── Spawns Python worker subprocess
            ↕ WebSocket + MessagePack
Python Worker (this repo)
    ├── Node discovery & execution
    ├── Local providers (HuggingFace, MLX)
    └── Media processing (ffmpeg, PIL, numpy)
```

## Worker Protocol

The worker speaks a msgpack message protocol to the TS server (bridge protocol **v6**, `nodetool.worker.BRIDGE_PROTOCOL_VERSION`). Two transports carry the same messages:

- **WebSocket** (default): `python -m nodetool.worker --host 0.0.0.0 --port 8787`. Without `--port` the worker binds a free port (`NODETOOL_WORKER_PORT`, default `0`). Do not use 7777, the NodeTool server's own default port. Each message is one binary msgpack frame. On startup the worker prints `NODETOOL_WORKER_PORT=<port>` to stdout (the only thing on stdout). If `NODETOOL_WORKER_TOKEN` is set, the opening handshake must carry `Authorization: Bearer <token>` (constant-time compare, rejected with 401 before any frame); unset means open, for local/dev use.
- **stdio** (`--stdio`): same msgpack payloads with 4-byte big-endian length-prefixed framing over stdin/stdout, for parent processes that spawn the worker directly.

Frames are capped at 256 MiB by default (`NODETOOL_BRIDGE_MAX_FRAME_SIZE`). Unknown msgpack extension types decode to `None` rather than erroring. Binary data (images, audio, model files) travels as native msgpack `bin` values — there is no base64.

### Message envelope

Every message is a map: `{"type": string, "request_id": string, "data": object}`. Requests carry a unique `request_id`; every frame the worker sends back echoes it, so concurrent requests multiplex safely over one connection. The worker replies with frames of type:

| Frame type | Meaning |
|------------|---------|
| `result` | Terminal success — one per request |
| `error` | Terminal failure — `data: {error, traceback}` |
| `progress` | Streamed progress updates (zero or more, before the terminal frame) |
| `chunk` | Streamed payload items (streaming execution, token streams, TTS audio) |
| `comfy.event` | ComfyUI execution events streamed during `comfy.execute` — `data: {event, prompt_id, ...}` |

`cancel` (`{type: "cancel", request_id: <id-of-in-flight-request>}`) requests cooperative cancellation of a running request; it produces no reply of its own — the cancelled request emits its own terminal frame.

### Core messages

| Request | Data | Response |
|---------|------|----------|
| `discover` | – | `discover` frame: `{protocol_version, nodes: [metadata], load_errors}` |
| `worker.status` | – | `{protocol_version, node_count, provider_count, namespaces, load_errors, transport, max_frame_size, comfy: {enabled, url}}` |
| `execute` | `{node_type, fields, secrets, blobs}` | `progress`\* then `result {outputs, blobs}` |
| `execute.stream` | same as `execute` | each yielded item as `chunk {outputs, blobs}`; empty `result` terminates the stream |
| `cancel` | – (uses `request_id`) | none |

For `execute`, `fields` are the node's property values; `blobs` maps field names to binary inputs (written to temp files and resolved as asset refs); `secrets` are forwarded to the node's context. Binary outputs come back in the result's `blobs` map.

### Provider messages (`provider.*`)

Local-compute providers (HuggingFace local, MLX) registered in this Python environment:

| Request | Data | Response |
|---------|------|----------|
| `provider.list` | – | `{providers: [{id, capabilities, required_secrets}]}` |
| `provider.models` | `{provider, model_type}` | `{models}` |
| `provider.generate` | `{provider, model, messages, tools?, max_tokens?, temperature?, top_p?, response_format?}` | `{message}` |
| `provider.stream` | same as `provider.generate` | `chunk {type: "chunk", content, done}` / `chunk {type: "tool_call", id, name, args}`, then `result {done: true}` |
| `provider.text_to_image` | `{provider, params}` | `result {blobs: {image}}` |
| `provider.image_to_image` | `{provider, image, params}` | `result {blobs: {image}}` |
| `provider.text_to_video` | `{provider, model, prompt, ...}` | `result {blobs: {video}}` |
| `provider.text_to_audio` | `{provider, model, prompt, ...}` | `result {blobs: {audio}}` |
| `provider.tts` | `{provider, model, text, voice?, speed?}` | `chunk {blobs: {audio}}`\* then `result {done: true}` |
| `provider.asr` | `{provider, model, audio, language?, ...}` | `{text, ...}` |
| `provider.embedding` | `{provider, model, text, dimensions?}` | `{embeddings}` |

`provider.stream` and `provider.tts` honor `cancel`.

### Model cache messages (`models.*`)

HuggingFace cache management on the worker's `HF_HOME`:

| Request | Data | Response |
|---------|------|----------|
| `models.list_cached` | – | `{models: UnifiedModel[]}` |
| `models.download` | `{repo_id, path?, allow_patterns?, ignore_patterns?}` | `progress {status: start/progress/completed/cancelled/error, downloaded_bytes, total_bytes, ...}`\* then `result {repo_id, status}` |
| `models.delete` | `{repo_id}` | `{deleted}` |

Tokens are resolved on the worker; client-supplied tokens are ignored. Downloads honor `cancel`.

### ComfyUI proxy messages (`comfy.*`)

When a ComfyUI server is co-located (`COMFYUI_URL`, advertised via `worker.status.comfy`), the worker proxies it — see [docs/comfy-proxy.md](docs/comfy-proxy.md) for frame-level shapes:

| Request | Data | Response |
|---------|------|----------|
| `comfy.execute` | `{workflow, blobs?, previews?, include_temp?, timeout?}` | `comfy.event` frames (`queued`/`executing`/`progress`/`node_output`/`preview`/…), then `result {prompt_id, status, outputs, blobs}` |
| `comfy.queue` | – | `{queue_running, queue_pending}` |
| `comfy.interrupt` | – | `{interrupted}` |
| `comfy.cancel` | `{prompt_id}` | `{cancelled, prompt_id}` |
| `comfy.upload` | `{data, filename?, kind?, subfolder?, overwrite?, original_ref?}` | ComfyUI upload response `{name, subfolder, type}` |
| `comfy.view` | `{filename, subfolder?, type?}` | `{filename, content_type, data}` |
| `comfy.object_info` | `{node_class?}` | `{object_info}` |
| `comfy.system_stats` | – | VRAM/RAM/device info |
| `comfy.free` | `{unload_models?, free_memory?}` | `{freed}` |
| `comfy.status` | – | `{enabled, url, reachable, system_stats?, queue_remaining?}` |
| `comfy.models.download` | `{folder, filename?, force?, source: {type: huggingface/url, ...}}` | `progress`\* then `result {status, path, size_bytes}` |
| `comfy.models.list` | – | `{models_dir, models: [{folder, filename, size_bytes}]}` |
| `comfy.models.delete` | `{folder, filename}` | `{deleted}` |

Input blobs are uploaded to ComfyUI and spliced into the workflow wherever a `"blob:<key>"` placeholder appears; file outputs are fetched from ComfyUI and returned as result blobs. `comfy.execute` and `comfy.models.download` honor `cancel`.

### Protocol version history

| Version | Changes |
|---------|---------|
| 1 | Initial protocol: `discover`/`execute`/`cancel` + `provider.*`, msgpack framing |
| 2 | `models.*` HuggingFace cache management |
| 3 | `comfy.*` ComfyUI proxy, `comfy.event` frame type, `comfy` capability block in `worker.status` |
| 4 | Optional execution identity fields, `job.start`/`job.end` run boundaries, `models.evict`, and node VRAM requirements |
| 5 | Optional `chunked-v1` result blob transfer with integrity checks, image-configured `models.prepare` adapters, and download activity telemetry |
| 6 | `provider.reference_to_video` command-adapter requests with staged reference media and ordered image/video path lists |

The TS bridge declares the minimum version it can speak; a worker reporting a lower `protocol_version` in `discover`/`worker.status` is rejected.

## External Node Packages

- **nodetool-huggingface** — HuggingFace model integrations + local provider
- **nodetool-mlx** — Apple Silicon optimized nodes + MLX provider (macOS on Apple Silicon only)
- **nodetool-wan2gp** — Wan2GP video generation through a Wan2GP server

Replicate, FAL and ElevenLabs integrations are TypeScript packs in the
[NodeTool](https://github.com/nodetool-ai/nodetool) repository.

## Development

Use the `nodetool` env from the quick start:

```bash
pytest -q          # tests
ruff check .       # lint
make typecheck     # ty type check (uses uv run)
```

`uv sync`, `uv run` and the `make` targets use a separate `.venv` pinned by
`uv.lock`, as CI does (`uv sync --locked --all-extras --dev`). After changing
dependencies in `pyproject.toml`, run `uv lock` and commit `uv.lock`.

## License

[AGPL License](LICENSE)

## Learn More

- [NodeTool Website](https://nodetool.ai)
- [Discord Community](https://discord.gg/WmQTWZRcYE)
