Metadata-Version: 2.4
Name: verda-io
Version: 0.1.0
Summary: Python worker framework for the Verda inference orchestrator
Project-URL: Homepage, https://github.com/verda-cloud/inferno
Project-URL: Repository, https://github.com/verda-cloud/inferno
Project-URL: Documentation, https://github.com/verda-cloud/inferno#readme
Project-URL: Issues, https://github.com/verda-cloud/inferno/issues
Author: Verda
Maintainer-email: Jaakko Varjo <jaakko@verda.com>
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: inference,ml,orchestrator,unix-socket,verda,worker
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: MacOS
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Requires-Dist: msgpack>=1.0.0
Description-Content-Type: text/markdown

# Verda IO

Python worker framework for the [Verda Inference Orchestrator](https://github.com/verda-cloud/inferno).

## Installation

```bash
pip install verda-io
```

Or with [uv](https://docs.astral.sh/uv/):

```bash
uv add verda-io
```

## Quick Start

Create a worker with initialization and prediction functions:

```python
import verda_io

@verda_io.initialize
def setup():
    """Called once at startup. Load your model here."""
    global model
    model = load_my_model()

@verda_io.predict
def predict(payload: bytes) -> bytes:
    """Called for each inference request."""
    result = model(payload)
    return result
```

Run it with the Verda server:

```bash
verda-io run server -c config.yaml
```

Or run the worker directly:

```bash
python -m verda_io main.py --socket /tmp/verda-io.data.sock --id worker-1
```

## Ordered Initialization

When your startup sequence requires specific ordering, use `@verda_io.initialize` with an order parameter. Functions run in ascending order:

```python
import verda_io

@verda_io.initialize(0)
def load_tokenizer():
    global tokenizer
    tokenizer = AutoTokenizer.from_pretrained("model-name")

@verda_io.initialize(1)
def load_model():
    global model
    model = AutoModelForCausalLM.from_pretrained("model-name")

@verda_io.initialize(2)
def warm_up():
    model.generate(tokenizer("warm up", return_tensors="pt").input_ids)
```

Using `@verda_io.initialize` without an argument defaults to order 0. Multiple functions with the same order run in registration order.

## Streaming Responses

Return a generator from your predict function to stream results:

```python
@verda_io.predict
def predict(payload: bytes) -> bytes:
    for token in model.generate_stream(payload):
        yield token.encode()
```

## API

- `@verda_io.initialize` - Register a startup function (called once, supports ordering)
- `@verda_io.predict` - Register the prediction function (called per request, exactly one required)
- `verda_io.run()` - Start the worker loop programmatically

## How It Works

The verda_io package connects to the Verda server via Unix domain sockets using a binary protocol (MessagePack + 12-byte headers). Initialization functions run once at startup in order, and `@verda_io.predict` is called for each inference request routed by the server.

## Requirements

- Python 3.10+
- Linux or macOS (Unix domain sockets required)

## License

Apache License 2.0
