Metadata-Version: 2.4
Name: outerproduct-sdk
Version: 0.1.17
Requires-Dist: adbc-driver-flightsql[dbapi]>=1.8,<2
Requires-Dist: cloudpickle==3.1.2
Requires-Dist: obstore>=0.11,<1
Requires-Dist: pydantic>=2.13.4,<3
Summary: High-level OuterProduct SDK for runs, images, data, and Unity Catalog.
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM

# OuterProduct SDK

`outerproduct-sdk` is the sole public Python package for OuterProduct. It owns
the Workspace API, runtime serialization, Files, Unity Catalog adaptation,
Flight SQL, and UC-governed object storage. `OuterProductClient` is the only
public client and owns all UC and non-UC operations for one automatically
selected Workspace.

## Natural workspace API

`op.init()` reads `OUTERPRODUCT_API_KEY` and `OUTERPRODUCT_BASE_URL`, ensures
`workspaces/default`, and returns an already-scoped client. Pass `workspace=`
to select another Workspace; callers never create or unwrap a second client.

```python
import outerproduct_sdk as op

client = op.init()
catalogs = client.list_catalogs()

image = op.Image.debian_slim(python_version="3.12").with_uv_pip_install(
    "numpy==2.3.2", "polars>=1.33"
)
hardware = op.HardwareSpec(provider=op.ExecutionProvider.MODAL, cpu=1)


async def add(
    runtime: op.OuterProductClient,
    left: int,
    right: int,
) -> int:
    return left + right


async def add_twice(
    runtime: op.OuterProductClient,
    left: int,
    right: int,
) -> int:
    pending = await runtime.run(fn=add, args=(left, right))
    first = await pending.wait()
    second = await runtime.run(fn=add, args=(first.result(), right))
    return (await second.wait()).result()


run = await client.run(
    fn=add_twice, args=(20, 22), image_spec=image, hardware_spec=hardware
)
result = (await run.wait()).result()

await client.write_file("results/answer.txt", str(result).encode())
answer = await client.read_file("results/answer.txt")
```

`Image` is an immutable recipe, constructed without network access. Use
`Image.from_registry("python:3.12-slim")` for a registry base, or
`Image.debian_slim(python_version="3.12")` to include managed Python. Ordered
`with_uv_pip_install(...)` steps declare dependencies.

`run()` accepts the function, arguments, image recipe, and hardware separately.
It returns a durable `Run` while preparation and execution proceed. On first
use, the control plane pins the base digest and hashes the recipe internally.
A temporary builder installs dependencies and publishes an immutable snapshot;
concurrent runs share preparation, and subsequent runs reuse the cached result.
Preparation status appears on the Attempt. Workers start with the prepared files.

Natural functions are async and receive their workspace client as the first
parameter. Child `runtime.run()` calls inherit omitted image and hardware
specifications. `await run.wait()` returns terminal state; `run.result()` decodes
its output. Pass `sources=`, `volumes=`, `environment_secrets=`, and
`retry_strategy=op.RetryStrategy(max_attempts=3)` directly to `run()`.

`max_attempts` includes the first attempt and defaults to 1 (no retries).
Retries preserve the function, arguments, and execution requirements; application
failures and worker loss share the Call's attempt budget. Each child selects its
own strategy. Cancellation is never retried, and parent replay does not reset
child budgets. Attempts can repeat external side effects, so retryable functions
should make those effects safe to repeat.

Unity Catalog CRUD remains part of the flat Workspace API, including on the
client injected into managed computations. The flattened methods retain the
generated Unity Catalog signatures for static type checkers. Temporary
table/volume/path/model credentials, Files, Flight SQL,
and runs are flat Workspace methods too. JSON literals, objects implementing
the SDK serialization contract, and Pydantic models can cross run boundaries.

## Workspace files

`op.init()` creates a Python `FileStore` automatically. On the first file
operation it reads the workspace's existing JSON metadata and derives the
`workspace-files` sibling of its managed storage root. It uses the client's
UC credential-vending methods and the existing `obstore` credential providers.
Cloud file bytes travel directly between the SDK process and object storage.

```python
import outerproduct_sdk as op

client = op.init()
await client.upload_file("/models/weights.bin", "weights.bin")
await client.download_file("/models/weights.bin", "downloaded.bin")
entries = await client.list_directory("/models")

await client.write_file("/notes.txt", b"hello")
contents = await client.read_file("/notes.txt")
```

Uploads and downloads handle chunking internally. Uploads use bounded multipart
concurrency and attempt to abort unfinished uploads on failure or cancellation.
Downloads write to a temporary file and replace the destination only after
completion. Directory listing returns a normal list and handles pagination
internally; `recursive`, `start_after`, and `max_results` control the selection.

Writes support atomic create-only and ETag preconditions. Hash preconditions
are rejected; a supplied content hash is checked against the bytes before
writing. Local filesystem stores do not persist content-type or hash metadata.
Empty directories use the server-compatible `.outerproduct-directory` marker.

Tests can inject an isolated store with
`op.init(_filestore=op.FileStore(MemoryStore()))`, importing `MemoryStore` from
`obstore.store`. Normal initialization requires no storage configuration.
Managed runtime clients continue to use their existing host file capability;
the direct file-transfer methods are available on clients created by `op.init()`.

## Serve a Hugging Face model

Create an inference endpoint with a Hugging Face repository and revision. The
endpoint downloads the model directly during initialization, loads it in vLLM,
and captures its CPU/GPU snapshot for subsequent restores.

```python
import outerproduct_sdk as op

client = op.init()
endpoint = await client.create_inference_endpoint(
    "Qwen chat",
    "Qwen/Qwen3-0.6B",
    revision="c1899de289a04d12100db370d81485cdf75e47ca",
    gpu="L4",
)
```

`revision` defaults to `main`; pass a full commit hash to select the same files
across separately created endpoints. Endpoint creation returns while
provisioning runs. Refresh the endpoint until its status is `READY`, then use
its `openai_base_url` for OpenAI-compatible requests. Download or model-loading
failures leave the endpoint `FAILED` with an error.

For authenticated downloads, create a Unity Catalog secret in the same workspace
before creating the endpoint:

```python
import os

client.create_secret(
    name="my_huggingface",
    values={"HF_TOKEN": os.environ["HF_TOKEN"]},
)
endpoint = await client.create_inference_endpoint(
    "Qwen chat",
    "Qwen/Qwen3-0.6B",
    gpu="L4",
    hf_token_secret="my_huggingface",
)
```

`hf_token_secret` defaults to `"hf_token"`. Endpoint creation reads the current
`HF_TOKEN` value from the named secret in this workspace and injects it into the
endpoint's environment. If the secret or key is absent, no token is supplied.
Catalog lookup errors fail endpoint
creation. The token is not stored in the endpoint resource or returned by the
API. Rotating the secret applies to newly created endpoints; existing endpoints
keep their deployed value.

## Storage boundaries

File operations are flat methods on the workspace client. Sources and volumes
belong to individual Calls and do not affect worker compatibility. Attach them
with `Function.with_source(...)` and `Function.with_volumes(...)`; child Calls
declare their own bindings.

`WorkspaceSources` accepts up to 64 absolute, normalized Workspace Files paths.
Each directory's entire contents merge into a temporary working directory; a
single file is placed under its basename. Hidden files, binary files, and empty
directories are preserved. Shared directories merge; duplicate files and
file/directory conflicts fail before user code executes. Repeated `.with_source`
calls append in order and return new handles.

The worker downloads sources after claiming the Call and before deserializing
the callable. The working directory becomes `cwd` and is prepended to `sys.path`
and `PYTHONPATH`. Files are read at execution time using the Call's credentials;
these references are live reads, not immutable snapshots or local file uploads.
There is no package detection or dependency resolution. Install dependencies
through the Image as usual.

`.with_volumes` accepts up to 16 bindings from absolute container destinations
to `catalog.schema.volume` names. Duplicate, overlapping, or already occupied
destinations fail. Volumes are downloaded using credentials for the current
Call; these are temporary directories, without write-back to storage.
Source files and volume destinations are removed on completion, failure, or
cancellation, and the previous working directory and Python paths are restored.
Children inherit neither sources nor volume bindings. Configured handles retain
their client; construct child handles through the injected `runtime` client.

Use `store_from_url`, `store_from_volume`, `store_from_table`,
`store_from_path`, or `store_from_model_version` for refresh-aware
UC-governed object stores.
`download_s3_prefix(client, prefix, target)` securely streams such a prefix
into a local directory with bounded concurrency.

## Packaging

The published distribution is one wheel with one native extension. Internal
serialization and UC object-store Python sources are vendored under
`outerproduct_sdk._vendor`; the wheel has no dependency on separately
published OuterProduct client packages. Third-party runtime dependencies remain
ordinary wheel dependencies.

