Metadata-Version: 2.5
Name: kestrel-feature-workflows
Version: 0.6.0
Summary: First-class agent workflow orchestration primitive for Kestrel Sovereign
License-Expression: Apache-2.0
License-File: LICENSE
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Requires-Python: <3.15,>=3.11
Requires-Dist: aiosqlite>=0.21.0
Requires-Dist: jsonschema>=4.20
Requires-Dist: kestrel-sovereign-sdk<1,>=0.36.0
Requires-Dist: kestrel-sovereign<1,>=0.53.3
Provides-Extra: test
Requires-Dist: prometheus-client>=0.20; extra == 'test'
Requires-Dist: pytest-asyncio>=1.1.0; extra == 'test'
Requires-Dist: pytest-timeout>=2.3.1; extra == 'test'
Requires-Dist: pytest>=8.0.0; extra == 'test'
Description-Content-Type: text/markdown

# kestrel-feature-workflows

First-class agent workflow orchestration for Kestrel Sovereign.

The package also installs a host-scoped `WorkflowsHostFeature` that owns the
durable, engine-neutral operator run plane. It registers the public SDK
`RunService` at host scope as `workflows.runs` version `1.0.0`, persists
accepted launches and their recovery outbox in the host database, and exposes
a fenced producer lifecycle for installed execution features. The run plane
does not add HTTP routes or engine-specific fields.

Absolute artifact links fail closed unless their exact HTTPS origins are
listed in the package-owned `KESTREL_WORKFLOWS_ARTIFACT_ORIGINS` environment
variable as a JSON array, for example
`["https://artifacts.example.com","https://cdn.example.com:8443"]`. Entries
must be exact HTTPS origins: paths, queries, fragments, credentials, and
non-HTTPS URLs are rejected. The value is limited to 16 KiB, 100 origins, and
2,048 characters per origin. Canonical `/authorized/artifacts/<artifact_id>`
links remain same-origin and need no allowlist entry.

Alternate hosts may instead provide the existing
`workflows.artifact_origins` context mapping. If both sources are present,
their normalized origin sets must be identical or host-feature startup fails;
otherwise the environment source is used when set. Malformed values fail
startup without logging their contents.

Workflows is orchestration **on top of** the existing Signal Dispatcher. Stages
dispatch signals (`ACTION` / `ARTIFACT` / `COGNITION`); durability, dedupe,
locks, causation tracking, retention, and redaction all come from the
dispatcher. Workflows adds what the dispatcher lacks: multi-stage sequencing and
fan-out, per-stage gates (including an adversarial `red_team_clear` gate),
reverse-order saga compensation, a cancellation barrier, DID-signed versioned
workflow definitions, and constitutional-boundary enforcement.

Execution is lease-owned: runnable runs and open stage/compensation attempts
are atomically claimed by an executor and heartbeated. Recovery reuses the
persisted attempt idempotency key and dispatch linkage; it advances confirmed
effects, never reruns completed stages, and fails safely when an external
effect cannot be determined. This provides exactly-once workflow transitions
and at-most-once effects where downstream idempotency supports it, not a
blanket exactly-once guarantee for arbitrary external systems.

### Declared runtime triggers

`WorkflowSpec.triggers` is honored at runtime for `manual`, `cron`, and
`signal_source` declarations. Manual behavior is unchanged. A non-manual
definition is rejected before persistence when its static trigger parameters
do not satisfy `params_schema`, its signal source is not registered, or the
required released Core capability is absent.

Cron declarations are registered only through SchedulerFeature's public
`schedule_add` / `schedule_list` / `schedule_remove` contract. They run in UTC
with `misfire_policy="skip"`; each callback must carry Scheduler's public,
non-forgeable execution identity. The scheduled occurrence time, not callback
time, is part of the deterministic workflow launch identity. Signal-source
declarations use SignalDispatcher's public agent-scoped durable consumer
registration and claim/lease/ACK/NACK lifecycle. Provider `source_event_id`
deduplication and the normalized durable event ID make redelivery converge on
one workflow run. A versioned agent/source boundary is captured before the
definition and desired registration commit, so retained history does not
launch retroactive runs while post-commit events survive registration/restart
windows.

The package persists an owner-and-live-dispatcher-scoped desired-state ledger
for every public registration. Definition writes, startup, soft re-enable,
revoke, feature disable, and feature removal reconcile that ledger. Retirement
intent is committed before the external resource is removed or deactivated;
publication identity is committed before SignalDispatcher registration and
Scheduler rows are adopted from their exact public arguments/idempotency key
after an ambiguous add result. This closes the supported cross-store crash
windows without reading Scheduler or SignalDispatcher private tables and
without claiming a transaction across their stores.

Signal trigger delivery is drained in bounded pages by the feature's existing
supervised lease-recovery cadence; declared triggers add no second polling
task. SchedulerFeature remains the cron wake owner and SignalDispatcher remains
the normalized-signal/durable-delivery owner. The default cadence can add up to
roughly ten seconds of signal-trigger launch latency. A trigger run persists
its registration, delivery/event or schedule identity, deterministic
idempotency key, and causation evidence before the transport is ACKed or the
Scheduler callback succeeds. A crash after that handoff is recovered through
the ordinary workflow run lease.

Signal events do not implicitly become workflow input. Only the trigger's
signed, static `params` are passed as `run.params`; triggering event data is
not projected into `run.params`. This includes its payload, caller identity,
and event identity. Event and delivery identifiers remain available as run
causation and inspection evidence. A future payload mapping must be an explicit
signed, schema-validated declaration; this release never merges untrusted event
data into stage parameters.

`workflow_list_definitions` includes registration state and
`workflow_inspect_definition` returns `trigger_health`, lifecycle state, last
error, and the last delivered event/run identity. Unhealthy registration or
cleanup is returned as a partial tool result rather than hidden. Owner DID,
live agent/dispatcher DID, definition/version/hash, and public resource identity
are checked again at every callback and delivery; a scope mismatch is never
reassigned to the current user.

`workflow_inspect_retention` exposes only the live owner's durable retention
debt. A terminal run missing its exact owner/name/version definition is retained
until that signed definition is restored; the next sweep clears the receipt
set-wise and applies the restored retention policy. This avoids guessing a
retention period or deleting another tenant's evidence.

See [the trigger architecture and migration notes](docs/architecture/WORKFLOW_TRIGGERS.md)
for identities, crash windows, and operational behavior.

Workflows persists the public durable-admission disposition and terminal
`SignalResult` for each await effect. Core 0.53.3 has no public historical
effect-status lookup, so a process loss after admission but before that result
is persisted remains durably `COMPENSATING` unless the host supplies
`workflow_effect_status_resolver` backed by authoritative public effect
evidence. Workflows never queries Core's private `signal_log` table to guess.

### Durable `await_signal` gates

An effect stage may use an `await_signal` gate to dispatch once, then wait for
an authenticated Core signal without retaining an asyncio task. The gate requires a
registered source, absolute timeout, `matcher_version: 1`, bounded `all`/`any`
matcher (`eq`, `in`, `exists` over JSON Pointers), and optional
sanitized-payload projection. Only
the closed `gate.passed` branch is permitted: its true target is the signal
path and its false target is the timeout path. Workflows persists the consumer,
tenant/owner scope, frozen context operands, deadline idempotency key, and
terminal evidence. Immediately before dispatching the external stage effect,
Workflows captures Core 0.53.3's public `DurableSourceBoundary` for the exact
agent/source scope and commits its complete versioned record in the wait row.
The row remains in `boundary_captured` setup state until effect admission; a
restart reuses it and never captures a later replacement. A claimed delivery
and the scheduler deadline race through one store transaction, so a late or
duplicate delivery can be audited but cannot replay the effect or graph
transition.

Core 0.53.3 provides the durable boundary and delivery lifecycle, but its
`DurableSignalEvent` does not persist an immutable, authenticated receipt of
the sanitizer and effective trust policy that applied at ingress. A sanitizer
registered today is not proof about a retained historical event. Therefore
Workflows 0.6.0 fails closed before the await effect dispatches unless the host
provides `workflow_await_signal_event_trust_verifier`, backed by a newer public
Core per-event attestation contract. Every delivered receipt is checked against
the event ID, dispatcher agent DID, source, and source sequence. The required
host verifier authenticates the receipt and validates its policy epoch against
the host's ingress-policy history; Workflows can only require a non-empty epoch
because Core 0.53.3 exposes no public policy-epoch provider to pin
independently. Workflows does not infer this proof from current registration
and does not inspect private Core tables. ACTION delivery remains ineligible
because it bypasses sanitization.

The host must also provide
`workflow_await_signal_privacy_transition_lock` through a public integration
surface. Core 0.53.3 keeps its corresponding agent lock private, so Workflows
does not reach into it; without the injected public lock the gate fails closed
before effect dispatch.

Core 0.53.3 serializes boundary capture, signal commit, and registration
backfill through one durable agent/source sequence domain on SQLite and
PostgreSQL. An event is eligible only when its `source_sequence` is strictly
greater than the persisted boundary sequence. `committed_at`, activation time,
delivery identity, and registration time are never *source-boundary*
eligibility fallbacks. Deadline eligibility is separate: a matching event wins
only when Core's linearized `committed_at` is strictly before the persisted
deadline; a commit exactly at the deadline belongs to timeout. Polling time
never changes that result. Timeout reconciliation gives publicly visible,
nonterminal pre-deadline deliveries a fixed 120-second grace after the
persisted deadline—two complete Core delivery-lease periods—to settle or
redrive. The grace is absolute and survives restart; reconciliation never
extends it. After it expires, timeout may win even when a source remains
disabled/unregistered, another executor still holds the lease, or Core's
public 1000-row inspection page is full. `deadline_won` evidence records every
visible unsettled pre-deadline delivery ID and whether that page hit the cap.
Malformed public rows without usable delivery IDs and failed initial drains
are represented by redacted synthetic unsettled identifiers; they receive the
same bounded grace and cannot bypass timeout liveness.
Core's public durable-consumer registration has no boundary field, so backfill
may materialize retained pre-boundary deliveries. Workflows validates every
public delivery against its persisted boundary, terminally NACKs a claimed
pre-boundary lease, and continues draining so a later post-boundary event is
not stranded. If Core claims the post-boundary row first, consumer cleanup may
terminalize the older row without creating Workflows evidence. A legacy row
without the complete boundary stays inspectable and cleanable after the
additive migration but fails closed for publication and resolution, including
on PostgreSQL.

Workflows uses a durable publication lease plus an `orphan_guard` tombstone
around the separate Core and Scheduler calls. Cancellation and force-abort
serialize with local publishers, remove exact returned resources after
ownership loss, and retain an unpurgeable retry intent after an interrupted
call whose eventual commit cannot be disproved. This is deliberately not
described as a cross-store atomic transaction.

Exact session selectors isolate runs by default; agent-wide broadcast is a
signed opt-in for INTERNAL events within the same user-agent tenant. Public,
external, and A2A-relayed events cannot win an agent-scoped wait; A2A authority
remains a separate surface. Recovery reads bounded owner/tenant/status-scoped
SQL pages through durable, wrapping cursors, quarantines malformed rows with
redacted structural audit evidence, and takes at most one public Scheduler
snapshot per pass.
Environmental registry reloads NACK for unbounded redelivery; immutable
scope mismatches ACK. Volatile, anonymized, and deidentified privacy transitions
are checked under Core's transition lock and atomically fence open waits. The
deadline winner takes the same lock and rechecks privacy before its row-locked
CAS. Timeout CAS eligibility and retention cleanup use database time; signal
arrival eligibility uses Core `committed_at`. Terminal recovery converges the
Core consumer and every related live Scheduler task while preserving fired
terminal one-shots as Scheduler audit history; purge waits for both cleanup
facts. A retention sweep supplied with the live dispatcher DID never performs
cleanup for an older/foreign dispatcher tenant: after retention expires it
persists a redacted stranded-resource receipt, visible through
`list_stranded_await_signal_retention`, and then purges the run. Omitting live
dispatcher context remains fail closed and lets outstanding cleanup pin the
run. Timeout wins compensate only the completed
await effect before the false branch proceeds; earlier PASS or irreversible
stages remain part of the live workflow. Shutdown drains boundedly and retains
unknown-effect evidence and leases for recovery when a handler resists
cancellation.

Delivery progress is currently periodic Workflows reconciliation, not a Core
event-driven callback. The feature reconciles once after initialization and
then every `min(lease_seconds / 3, 10 seconds)` (with a 50 ms floor), so the
default 30-second lease gives up to roughly 10 seconds of wake latency plus
processing time. Scheduler one-shots provide lower-latency timeout wakeups, but
each reconciliation/restore pass also scans a fair bounded database-clock lane
and resolves overdue waits through the same winner CAS, so timeout liveness
does not depend on Scheduler callback delivery. Availability requires the
Workflows recovery worker, its database, and Core durable delivery APIs;
Scheduler list/remove availability is still needed to converge live task
cleanup while preserving terminal Scheduler rows as audit history. A future
Core event-driven caller may reduce signal latency but is not part of this
release.

## Installation

```bash
uv pip install kestrel-feature-workflows
```

The package registers `WorkflowsFeature` through `kestrel_sovereign.features`
and `WorkflowsHostFeature` through `kestrel_sovereign.host_features`.

## Operator Runs API and console

Workflows 0.6.0 exposes its durable, engine-neutral run plane at
`/api/workflows/runs`. Authenticated operators can page and filter run records,
inspect stages, numbered attempts, typed external job correlations, and
artifact metadata, apply pause/resume/cancel/retry controls with idempotency and
sequence preconditions, and follow one authorized run through a polling-backed
SSE stream. Cross-tenant and absent identifiers share the same not-found
response, and artifact dereference returns metadata only.

`WorkflowsHostFeature` also contributes the capability-gated **Runs** panel to
the Console from package-owned JavaScript and CSS. The panel uses authenticated
streaming with polling fallback, renders generic SDK fields only, and offers an
optional correlation handoff to an installed Observability panel.

An injected `workflows.operator_context_resolver` remains authoritative for
Castle and other multi-tenant deployments. Otherwise Workflows resolves the
current host-scoped `claws.catalog>=1.3` service for each request and asks its
public operator-context resolver to authorize OAuth and other authenticated
solo operators. A present Claws service is authoritative: policy denial or an
invalid resolver fails closed and cannot fall through to broader local rights.
Generic Workflows hosts without a compatible Claws service retain the existing
zero-config fallback, which grants access only to the Sovereign API-key caller
and requires an injected tenant resolver. A present 1.2 catalog without the 1.3
operator policy is not considered absent and fails closed. The selected
resolver is responsible for user, tenant, boundary, capability, action, and
expiry policy.

The HTTP and Console surfaces intentionally do not launch runs. Launch remains
the feature-owned `RunService.launch_run` boundary used by installed execution
features, where source attribution, target boundaries, capabilities, tenancy,
and idempotency are authorized before acceptance. No Talon-, Eye-, Flight-,
repository-, command-, or filesystem-specific fields are part of this API.

## Development

```bash
uv sync --extra test
uv run --extra test pytest -q
npm test
```

## Design

See `docs/architecture/WORKFLOWS_FEATURE_DESIGN.md` (v4.1) and
`WORKFLOWS_DEVELOPER_GUIDE.md` in the kestrel-sovereign repository for the full
design and the stage-to-signal reduction.
