Metadata-Version: 2.4
Name: utic-invocation-settings
Version: 0.3.0
Summary: Public library for consuming encrypted Unstructured plugin invocation settings (AES-256-GCM + RSA-OAEP-256 envelope, decrypt + cache).
Project-URL: Homepage, https://github.com/Unstructured-IO/utic-public-libs
Project-URL: Repository, https://github.com/Unstructured-IO/utic-public-libs
Project-URL: Documentation, https://github.com/Unstructured-IO/utic-public-libs/tree/main/libs/utic-invocation-settings#readme
Project-URL: Threat model, https://github.com/Unstructured-IO/utic-public-libs/blob/main/libs/utic-invocation-settings/docs/threat-model.md
Project-URL: Changelog, https://github.com/Unstructured-IO/utic-public-libs/blob/main/libs/utic-invocation-settings/CHANGELOG.md
Project-URL: Issues, https://github.com/Unstructured-IO/utic-public-libs/issues
Requires-Python: >=3.11
Requires-Dist: cryptography>=43.0.1
Requires-Dist: pydantic<3.0.0,>=2.12.5
Description-Content-Type: text/markdown

# utic-invocation-settings

Public library for **consuming encrypted Unstructured plugin invocation settings** — the plugin-side
half of the cellular-dataplane "settings in the invoke payload" design. It reads the v1 settings
envelope (RSA-OAEP-256-wrapped **AES-256-GCM**), decrypts **inside the plugin** at invoke time, and
caches only previously authenticated envelopes.

It is deliberately **self-contained on `cryptography` + `pydantic`** — no dependency on any
private-feed package — so it can be published to public PyPI and imported by external plugin authors.

Ships [PEP 561](https://peps.python.org/pep-0561/) type information (`py.typed`): the result type of
every call below is inferred, not `Any`.

## Why

Under the cellular dataplane, a shared pod may serve multiple tenants, so a plugin identity decrypts
settings routed to that plugin rather than a shared service handing out plaintext. Settings arrive as an
opaque ciphertext envelope; this library turns that envelope into a plain settings object, verifying
integrity and never logging secrets.

The wire format is the `Envelope` model in `utic_invocation_settings/envelope.py`: its field
constraints are the contract, and `Envelope.model_json_schema()` exports them as JSON Schema. The
producer (Secrets Provider / operator) emits exactly that shape; this library is the reference
consumer. What that format does and does not guarantee is written down in
[the threat model](https://github.com/Unstructured-IO/utic-public-libs/blob/main/libs/utic-invocation-settings/docs/threat-model.md)
— read it before you rely on an envelope for anything.

## See it work

[`docs/walkthroughs/envelope-cryptography/`](https://github.com/Unstructured-IO/utic-public-libs/blob/main/libs/utic-invocation-settings/docs/walkthroughs/envelope-cryptography/README.md) holds three runnable walkthroughs —
executable documentation, not production code. No cluster, no fixtures:

```bash
cd libs/utic-invocation-settings
uv run --no-sync python docs/walkthroughs/envelope-cryptography/round_trip.py
```

| | |
|---|---|
| [`round_trip.py`](https://github.com/Unstructured-IO/utic-public-libs/blob/main/libs/utic-invocation-settings/docs/walkthroughs/envelope-cryptography/round_trip.py) | Both sides in one file: settings in → sealed → settings out → refusals. |
| [`producer_side.py`](https://github.com/Unstructured-IO/utic-public-libs/blob/main/libs/utic-invocation-settings/docs/walkthroughs/envelope-cryptography/producer_side.py) | Builds an envelope with only `cryptography` and the standard library, then opens it with `resolve_settings`. A producer in another language depends on exactly this working. |
| [`consumer_side.py`](https://github.com/Unstructured-IO/utic-public-libs/blob/main/libs/utic-invocation-settings/docs/walkthroughs/envelope-cryptography/consumer_side.py) | Seals with the library, then walks the **core cryptographic path** by hand — not every check `resolve_settings` makes; the walkthrough README lists what it leaves out. |

They print the fixture settings at both ends with a digest beside them, and the envelope as JSON,
so the round trip is visible rather than asserted at you. The digest is the part that matters: it
shows the same *bytes* came back, not merely equal-looking values. Where an envelope is abridged for
width it is labelled as a display copy that will not open. The private key is never printed, and
neither is any value the library echoed out of a rejected settings document.

`tests/unit/test_walkthroughs.py` runs each file and checks it exits `OK` — each self-verifies, so a
walkthrough that stopped matching the library fails rather than misleading you. `tests/packaging`
asserts the directory is absent from the built wheel.

## Usage

Declare the settings your plugin needs, then resolve them from the envelope:

```python
from typing import Any

import pydantic
from utic_invocation_settings import resolve_settings


class MySettings(pydantic.BaseModel):
    api_key: str
    timeout_seconds: int = 30


def on_invoke(body: dict[str, Any]) -> dict[str, Any]:
    envelope = body["invocation_settings"]["dag_node_settings"]
    settings = resolve_settings(envelope, MySettings)  # raises if unusable
    return fetch(settings.api_key, timeout=settings.timeout_seconds)
```

`settings` is a `MySettings`, so `settings.api_key` type-checks and a typo does not.

That one call loads this pod's mounted workload key, decrypts, caches, and validates into
`MySettings`. Pass a pydantic model class, a `TypeAdapter`, any callable taking the settings mapping,
or nothing at all for the plain mapping (`resolve_settings(envelope)`). Each of those four is a
separate typed overload, so the inferred result is the model, the adapter's type, the callable's
return type, or `dict[str, Any]`. The envelope itself may be a raw JSON mapping or an already-parsed
`Envelope`.

### The input is one envelope

`resolve_settings` takes the envelope, not the request body. Where the envelope lives in your
payload, and what an absent one means, are your wire contract — this library holds the crypto and
knows nothing about `/invoke`. On the Unstructured plane the envelope is
`body["invocation_settings"]["dag_node_settings"]`, and a plugin that also needs a boot-time settings
file writes that branch itself:

```python
from collections.abc import Mapping
from typing import Any

_MISSING = object()   # NOT None: `null` is a value that arrived, not an absence


def sealed_field(body: Mapping[str, Any]) -> Any:
    """Exactly `invocation_settings.dag_node_settings`, or `_MISSING` if it was not sent."""
    container = body.get("invocation_settings", _MISSING)
    if container is _MISSING:
        return _MISSING
    if not isinstance(container, Mapping):
        return container            # present and not an object — a value, so let it fail below
    return container.get("dag_node_settings", _MISSING)


sealed = sealed_field(body)
if sealed is _MISSING:
    settings = load_boot_settings()               # nothing was sent: your trust decision
else:
    settings = resolve_settings(sealed, MySettings)   # anything else must survive this call
```

**Only a genuinely absent field may reach the fallback.** `null`, `{}`, a malformed envelope, one
sealed to another recipient, one that fails authentication, and one whose plaintext your model
rejects are all values that *arrived*; every one of them raises out of `resolve_settings`. Use a
missing sentinel, never `is None` — `.get("dag_node_settings")` collapses "not sent" and "sent as
`null`" into the same answer, and that single line is the whole vulnerability.

**Never fall back after an `InvocationSettingsError`.** Do not wrap the `resolve_settings` call in a
`try` that falls back on failure: that answers a request configured for one tenant with whatever
this pod booted with. The fallback belongs on the absence branch only, before the call.

Writing it out is the point. Deciding that an absent envelope may be answered from local defaults is
a **trust** decision, and one worth seeing at the call site: it lets anyone who can reach `/invoke`
choose your defaults over a tenant's sealed settings by omitting a field. This library used to own
that branch behind a policy enum; it no longer does, because it cannot see the body the decision is
about.

What has not moved: **an envelope that arrived and could not be used always raises.** Wrong
recipient, tampered, unparseable, unknown version, or a broken local mount — there is no path here
that answers a request configured for one tenant with whatever the pod booted with.

### Async handlers

**There is no async API, by design.** This library is CPU-bound: a resolve is an RSA unwrap, an
AES-GCM open, two SHA-256 digests, a JSON parse, and your model's validation. It opens no sockets and
makes no network calls. The only IO anywhere in the package is reading the workload-identity mount
(`tls.key`/`tls.crt`), and `WorkloadIdentity.load()` memoizes the result — so that is one pair of
small file reads per process per mount directory, not per request.

So an `async def` entry point would have nothing to await. Offering one would imply that awaiting is
natural because something blocks on IO, when the honest description is "this burns CPU for a couple
of milliseconds". Whether to move CPU work off your event loop is a judgement about *your* latency
budget, and `asyncio.to_thread` is the stdlib primitive for exactly that:

```python
async def on_invoke(envelope: dict):
    settings = await asyncio.to_thread(resolve_settings, envelope, MySettings)
```

This is not a formality — it is worth doing. A cold resolve stalls every other task on the loop for
~2.2 ms, and OpenSSL releases the GIL for the RSA operation, so a worker thread genuinely absorbs it
rather than merely relocating the stall. The technique is sound; it was the *interface* that was
wrong, since a wrapper around `to_thread` adds no capability a caller does not already have.

It **keeps the inferred result type**: `to_thread` is generic over a `ParamSpec`, so `mypy --strict`
infers `MySettings` above, including through bound methods (`resolver.resolve`). The typing-contract
test asserts this for every overload shape.

One note on cost: the thread hop is ~36 µs, and a warm resolve is ~51 µs — so if you resolve the same
envelope repeatedly, offloading buys less than the cold-path number suggests.

### Provenance, and configuring a resolver

`SettingsResolver(...)` is the configurable form (same relationship as `requests.get` to
`requests.Session`) when the cache TTLs and budgets, the clock, or the key loader need supplying. A
resolver constructs and privately owns its caches — configured by validated scalars
(`settings_ttl_seconds`, `key_ttl_seconds`, `settings_cache_max_bytes`, or the `DISABLED` sentinel),
never by injected cache instances — and identity is bound per resolver with no per-call override,
because a resolver's caches derive their authority from its identity.

`SettingsResolver.resolve_detailed(envelope)` returns the envelope's metadata alongside the plaintext:
`expires_at`, `credential_version`, and the authenticated `settings_digest` as a ready-made
`cache_key` for caller-side derived objects. There is no module-level `resolve_detailed`; reach the
shared instance with `default_resolver().resolve_detailed(envelope)`.

### Errors

Every failure (unknown format, missing key, RSA/GCM failure, digest mismatch, model rejection)
raises a subclass of `InvocationSettingsError` — it never returns partial or unverified plaintext.
Each class carries three class attributes so a host can map an outcome onto its own transport
without matching on messages:

- **`reason`** — a stable machine-readable code, unique across the taxonomy. The only part safe to
  switch on.
- **`blame`** — which participant to investigate (`CALLER`, `RECIPIENT`, `CONTENT`, `ROUTING`), and
  the single input to the HTTP mapping below.
- **`retry`** — a `RetryDisposition`: `NEVER`, `SAME_REQUEST` (replay these identical bytes; the
  fault is a local transient such as a Secret that has not been projected yet), or `NEW_INVOCATION`
  (replaying is futile, but the *platform* can compose or re-address an invocation that succeeds —
  explicitly **not** a client retry).

`retryable` remains available as a strictly narrower derived shim: it is `True` only for
`SAME_REQUEST`, and it is computed from `retry` so the two cannot drift.

**HTTP class comes from `blame`, by one rule:** `blame is Blame.CALLER` → **422**, anything else →
**5xx**. The line it draws is whether a different request would work. Settings that were required
and not supplied, or that arrived in a shape this contract does not allow, are the caller's to fix;
settings that were supplied and could not be *processed* are orchestration — the producer, this
pod's mounted identity, or the routing between them — which no request the caller composes can
repair.

| error | `reason` | `blame` | `retry` | HTTP |
|---|---|---|---|---|
| `MalformedEnvelopeError` | `malformed_envelope` | `CALLER` | `NEVER` | 422 |
| `UnsupportedFormatError` | `unsupported_format` | `CALLER` | `NEVER` | 422 |
| `KeyNotFoundError` | `recipient_mismatch` | `ROUTING` | `NEW_INVOCATION` | 5xx |
| `DecryptionError` | `decryption_failed` | `CONTENT` | `NEVER` | 5xx |
| `IntegrityError` | `integrity_mismatch` | `CONTENT` | `NEVER` | 5xx |
| `SettingsValidationError` | `settings_validation_failed` | `CONTENT` | `NEVER` | 5xx |
| `IdentityNotMountedError` | `identity_not_mounted` | `RECIPIENT` | `SAME_REQUEST` | 5xx |
| `IdentityUnreadableError` | `identity_unreadable` | `RECIPIENT` | `SAME_REQUEST` | 5xx |
| `IdentityMaterialError` | `identity_material_invalid` | `RECIPIENT` | `NEVER` | 5xx |
| `CertificateRequiredError` | `certificate_required` | `RECIPIENT` | `SAME_REQUEST` | 5xx |
| `IdentityConfigurationError` | `identity_configuration_invalid` | `RECIPIENT` | `NEVER` | 5xx |

The identity rows are where `retry` earns its keep: a Secret that has not been projected yet is
`SAME_REQUEST` (replay these identical bytes), while a mount that is present and *wrong* is `NEVER`
and pages someone. Flattening those together fails pods permanently on an ordinary startup race.

The classification rule, so a new class has an obvious answer rather than a judgement call: presence
faults are transient (`SAME_REQUEST`), content faults are permanent (`NEVER`), and addressing faults
need a new invocation.

`SettingsValidationError` additionally carries `issues`: a bounded tuple of `SettingsIssue`, each a
`code` drawn from pydantic's own closed error vocabulary and a `loc` whose string components must be
declared by your model's schema (anything else — a mapping key, a discriminator value — is
`<redacted>`, because for a settings payload those come from the input and the input is the secret).
The validator's own message is never reproduced: a custom validator is free to interpolate the
rejected value into it, and callers do.

### Where the key comes from

`WorkloadIdentity` reads `tls.key` (and `tls.crt` when present) from `$WORKLOAD_IDENTITY_DIR`, else
`$INVOCATION_SETTINGS_KEY_DIR`, else `/var/run/workload-identity`. When a certificate is mounted it
is the **anchor**: the `kid` comes from the certificate and the mounted key must match it. A `kid`
this pod does not hold yields `KeyNotFoundError` (investigate routing); a mount that is absent or
self-inconsistent yields `IdentityConfigurationError` (investigate this pod's Secret).

Where certificates are projected fleet-wide, make anchoring mandatory — with
`WorkloadIdentity(require_certificate=True)` or `$WORKLOAD_IDENTITY_REQUIRE_CERTIFICATE=true`, which
reaches pods that only ever call the module-level `resolve_settings`. A key-only mount then raises
`CertificateRequiredError` instead of silently self-anchoring on the key, which would make this pod
answer for a `kid` the producer never sealed to.

`IdentityConfigurationError` now has transient/permanent subclasses (see the table above): "not
projected yet" and "present and unreadable" are `SAME_REQUEST`, while "present and wrong" —
unparseable PEM, non-RSA key, key below the RSA-3072 floor, a certificate that does not match the
key — is `NEVER`, and is a paging condition rather than a backoff condition.

### Lower-level primitives

Still public, for a consumer that already holds an envelope or wants to own the orchestration:

```python
settings = decrypt_settings(               # decrypt + parse; the primitive itself never caches
    envelope,
    private_key_loader=load_private_key,
    limits=DEFAULT_LIMITS,                 # optional: tighten (never loosen) the size / JSON ceilings
)
```

Caching is not a property of the primitives — `open_envelope` and `decrypt_settings` retain nothing
and take no cache. A process that must bound or reuse recovered plaintext constructs a
`SettingsResolver`, which owns an authenticated, identity-scoped cache privately and re-authorizes
every hit; no cache instance crosses the API boundary in either direction.

The root API stops there on purpose. Cache-key derivation (`envelope.envelope_fingerprint`), the
authenticated-header byte layout (`envelope.protected_aad`) and envelope *production*
(`crypto.seal_settings`) are supported but live in their own submodules: re-exporting them would make
each one's representation a root-level compatibility contract.

## Security and cache notes

See [`docs/threat-model.md`](https://github.com/Unstructured-IO/utic-public-libs/blob/main/libs/utic-invocation-settings/docs/threat-model.md)
for the full threat model, and
[`docs/adr-0001-envelope-v2.md`](https://github.com/Unstructured-IO/utic-public-libs/blob/main/libs/utic-invocation-settings/docs/adr-0001-envelope-v2.md)
for the proposal that closes the gaps it documents.

- This is encryption and integrity for the recipient, not producer authentication. A holder of the
  recipient's public key can create an envelope, so deployments must deliver envelopes over an
  authenticated control-plane path.
- `expires_at` and `credential_version` are plaintext advisory metadata outside the authenticated
  header. Use them for scheduling/freshness hints, never authorization decisions. `expires_at` may
  only ever *shorten* a cache lifetime, so the worst a party who rewrites it can achieve is making
  this process decrypt more often.
- `settings_digest` is a public, unsalted SHA-256 value and can link identical settings or support
  low-entropy guess confirmation. Treat envelopes as sensitive metadata.
- **A cache is never authority.** Every hit — plaintext or content key — is reauthorized against the
  envelope's recipient before it is served, so sharing a cache between key loaders cannot leak
  across identities.
- The decrypted-settings cache is keyed by the full authenticated-envelope fingerprint; the AES-key
  cache is keyed by `(kid, encryption_key_digest, sha256(encrypted_key))`. Both keys are namespaced
  by the recipient `kid`. `clear()` is not a revocation barrier for work already decrypting
  concurrently.
- The plaintext cache stores a **deeply-immutable parsed snapshot**, and every hit rebuilds a fresh
  mutable tree from it, so two concurrent resolves can never alias one settings object. Its byte
  charge is an estimate of that parsed structure, measured per document — a parsed mapping runs
  2.6x-23.7x its JSON size depending on shape, so a constant multiplier would silently under-count.
- There is a **256 KiB cache-admission ceiling** (`MAX_CACHEABLE_SETTINGS_BYTES`) and an **8 MiB
  byte budget** (`DEFAULT_SETTINGS_CACHE_MAX_BYTES`). Settings larger than the admission ceiling
  still resolve — they simply pay the full cold cost every time and are never cached; nothing is
  rejected merely for failing *cache admission*, which is not a correctness property. This is
  distinct from the envelope's hard **1 MiB plaintext wire ceiling** (`MAX_PLAINTEXT_BYTES`, 4x the
  admission ceiling, with `MAX_CIPHERTEXT_CHARS` derived from it): an envelope whose plaintext would
  exceed *that* is refused at the boundary as a wire-contract violation the caller can act on. The
  two thresholds are deliberately separate decisions — "too big to retain" versus "too big to
  accept".
- For a genuine no-cache deployment, pass the `DISABLED` sentinel:
  `SettingsResolver(settings_ttl_seconds=DISABLED, key_ttl_seconds=DISABLED)` retains nothing between
  calls, at the price of the full ~2.2 ms cold cost on every invoke. It is a real no-cache, not a
  short TTL, and is compared by identity so no falsy `0`/`None`/`""` reaches the disabled branch by
  accident.

**Reporting a vulnerability.** Do not open a public issue. Report privately via the
[repository's Security tab](https://github.com/Unstructured-IO/utic-public-libs/security) (GitHub
private vulnerability reporting), or to the Unstructured maintainers through your existing support
channel. Include the package version and the envelope *shape* — never real ciphertext, settings
values, or key material.

## Develop

```bash
make install          # uv sync --locked
make test             # unit tests + coverage
make test-packaging   # build the wheel, install it clean, check metadata + consumer types
make check            # ruff + version consistency
```

`make test-packaging` builds the distribution once and tests that exact artifact in a fresh virtual
environment: version agreement across `pyproject.toml` / `uv.lock` / wheel metadata / installed
metadata, `py.typed` presence, the declared dependency set, the root public API, and a mypy
consumer-contract check that the documented result types are what a call site actually infers.

**Note:** published to public PyPI on merge to main (see the repository README).
