Metadata-Version: 2.4
Name: dataoad
Version: 0.1.0
Summary: Safe execution primitives for side-effecting agent actions
Author: Dataoad contributors
License-Expression: Apache-2.0
Keywords: agents,idempotency,distributed-systems,reconciliation
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Typing :: Typed
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: dev
Requires-Dist: build>=1.2; extra == "dev"
Requires-Dist: mypy>=1.10; extra == "dev"
Requires-Dist: pytest>=8.2; extra == "dev"
Requires-Dist: pytest-asyncio>=0.23; extra == "dev"
Requires-Dist: ruff>=0.5; extra == "dev"
Dynamic: license-file

# Dataoad

> Open-source execution contracts and failure testing for side-effecting AI
> agent actions.

A timeout does not prove an action failed.

A successful tool call does not necessarily prove the intended external effect
occurred.

```text
Naive retry
calls=2
charges=2

Dataoad
execution=UNKNOWN
verification=CONFIRMED
final=CONFIRMED
calls=1
charges=1
```

Dataoad provides safe execution primitives for agents and other software that
take real-world actions. If a charge, ticket, email, refund, or infrastructure
change may have happened before its response was lost, blindly retrying can
repeat the side effect.

Dataoad makes that uncertainty explicit:

```text
CONFIRMED != FAILED != UNKNOWN
```

`UNKNOWN` is allowed to remain `UNKNOWN`. The default policy never
automatically retries an ambiguous side-effecting action.

Dataoad is an early, local-first library and fault harness. It is not
production-ready, does not make arbitrary third-party APIs idempotent, and does
not guarantee exactly-once execution.

## Install

Dataoad requires Python 3.11 or newer. From a repository checkout:

```bash
python -m venv .venv
```

Activate the environment with `. .venv/bin/activate` on POSIX or
`.venv\Scripts\Activate.ps1` in Windows PowerShell, then install the package:

```bash
python -m pip install -e .
```

For tests and contributor tools, install the development extra instead:

```bash
python -m pip install -e ".[dev]"
```

## 30-second example

This fake provider commits a charge and then raises `TimeoutError`, reproducing
a lost response after mutation. Dataoad records `UNKNOWN`, makes no second
charge, and then uses a read-only verifier to reconcile the receipt.

```python
import asyncio

from dataoad import ActionRunner, ActionStatus, InMemoryLedger
from dataoad.testing import ChargeFault, FakePaymentProvider, PaymentVerifier


async def main() -> None:
    provider = FakePaymentProvider(charge_faults=[ChargeFault.TIMEOUT_AFTER_COMMIT])
    runner = ActionRunner(InMemoryLedger())
    request = {"customer_id": "customer_123", "amount_cents": 2_500}

    result = await runner.execute(
        "charge_customer",
        lambda: provider.charge(
            customer_id="customer_123",
            amount_cents=2_500,
            client_reference="order-123",
        ),
        request=request,
        idempotency_key="order-123",
        provider_reference="order-123",
        risk="financial",
    )

    assert result.status is ActionStatus.UNKNOWN
    assert provider.charge_calls == 1

    result = await runner.reconcile(result.receipt.action_id, PaymentVerifier(provider))
    assert result.status is ActionStatus.CONFIRMED
    assert provider.charge_calls == 1  # verification did not repeat the mutation


asyncio.run(main())
```

See [`examples/payment_timeout_after_commit.py`](examples/payment_timeout_after_commit.py)
for the complete naive-retry comparison and safety suite. The example is
included in repository checkouts and source distributions; it is not installed
as package data by the wheel.

For durable local receipts, give `ActionRunner` a file-backed `SQLiteLedger`
instead. Reopening the same database preserves completed receipts and their
idempotency claims across process restarts:

```python
from pathlib import Path

from dataoad import ActionRunner, SQLiteLedger

runner = ActionRunner(SQLiteLedger(Path("dataoad.sqlite3")))
```

## State semantics

The public outcome is an evidence statement, not merely a translation of an
exception:

| State | Exact meaning | Default mutation behavior |
| --- | --- | --- |
| `CONFIRMED` | There is sufficient evidence that the intended side effect happened. | Do not retry. |
| `FAILED` | There is sufficient evidence that the intended side effect did not happen. | Retry only when policy and adapter evidence explicitly allow it. |
| `UNKNOWN` | There is not enough trustworthy evidence to decide whether it happened. | Never automatically retry. |

An ordinary exception after the operation is invoked becomes `UNKNOWN`. This
includes timeouts, lost connections, and malformed responses whenever mutation
may already have occurred. `DefinitiveFailure` is different: raising it is a
strong assertion by the provider adapter that the intended side effect did not
happen. Use it only when that fact is known, not as a wrapper for a generic
provider error.

A normal return is treated according to the evidence contract chosen by the
caller:

- Without a verifier, Dataoad preserves the simple v0.1 behavior and trusts the
  operation response as authoritative evidence of occurrence. The receipt is
  `CONFIRMED` even if the returned Python value is falsey.
- With a verifier, a normal return establishes only that execution completed.
  Dataoad records direct external-effect evidence as `UNKNOWN`, always runs the
  verifier, and lets its bound evidence determine `CONFIRMED`, `FAILED`, or
  `UNKNOWN`.

Receipts preserve three related fields:

- `execution_status` describes what direct execution established.
- `verification_status` describes what independent read-back established.
- `final_status` is the current public confidence statement returned as
  `ActionResult.status`.

A timeout-after-commit can therefore have `execution_status=UNKNOWN`,
`verification_status=CONFIRMED`, and `final_status=CONFIRMED`. `PENDING` and
`COMPLETED` are separate internal claim-lifecycle states; `PENDING` is not a
fourth public outcome.

`CONFIRMED` means that sufficient evidence shows the intended effect happened
at least once. It does not prove that the effect happened exactly once or that
no duplicate exists.

Evidence is also aggregated across attempts. Once any attempt may have produced
an unverified side effect, a later independent `FAILED` attempt cannot erase
that uncertainty: the action remains `UNKNOWN`. A later authoritative
confirmation may establish `CONFIRMED`; an action-wide final-state verifier may
also resolve all relevant attempts explicitly.

`ActionResult.value` cannot be read for `FAILED` or `UNKNOWN` results. It raises
`UnresolvedActionError`, so ambiguous work cannot look like normal success.

## Retry policy

The default `RetryPolicy` performs one attempt and retries neither `FAILED` nor
`UNKNOWN`. A retry of `FAILED` requires remaining capacity under
`max_attempts`, `retry_on_failed=True`, and trustworthy evidence that no side
effect occurred. That evidence is either:

- a `DefinitiveFailure(..., retryable=True)` from the adapter; or
- `VerificationResult.absent(..., retry_safe=True)` from a verifier that
  establishes healthy, conclusive absence **and** proves the original request
  is terminal or fenced so it cannot commit later.

`VerificationResult.absent(...)` defaults to `retry_safe=False`. A healthy
read-back that shows nothing now remains `UNKNOWN` while the original request
could still be in flight. Set `retry_safe=True` only when the adapter can prove
terminal or fenced non-occurrence; that stronger result becomes `FAILED` and
may authorize retry when policy also allows it.

`retry_on_unknown=True` is available only as an explicit escape hatch and emits
a runtime warning. It can duplicate real-world side effects unless the
downstream operation is independently safe.

Direct calls to `RetryPolicy.should_retry(...)` validate their runtime inputs
before making any decision: `status` must be an exact `ActionStatus`,
`attempts` an exact positive integer, and `failure_is_retryable` an exact
boolean. Invalid values raise instead of falling through to a retryable branch.

The execution protocol owns this decision. An LLM or agent should not be asked
to guess whether an ambiguous mutation is safe to repeat.

## Idempotency semantics

An idempotency claim is scoped by `(action_name, idempotency_key)`. `request`
must include every input that can affect the intended side effect. Dataoad
stores its canonical SHA-256 hash rather than the raw request.

The request must be an exact native Python `dict` representing a strict JSON
object. Values may contain only `None`, exact `bool`, exact `int`, finite exact
`float`, exact `str`, lists, and dictionaries with string keys. Python-specific
representations such as tuples, enums, dataclasses, dates, datetimes, UUIDs,
custom containers, and scalar subclasses are rejected instead of being
silently coerced. Convert them explicitly at the call site when that conversion
matches the application's intended semantics.

For one shared ledger:

1. Same action, key, and request hash while the first claim is `PENDING` raises
   `ActionInProgressError`; the duplicate operation is not invoked.
2. Same action, key, and request hash after completion replays the existing
   receipt with `result.replayed=True`; the operation is not invoked again. If
   that receipt is `UNKNOWN` and a verifier is supplied, Dataoad attempts
   read-only reconciliation instead.
3. Same action and key with a different request hash raises
   `IdempotencyConflictError` before the new operation can run.
4. Concurrent contenders are reduced to one winning claim by the ledger's
   atomic conditional write.
5. The same key under a different action name is a different claim.

`provider_reference` is a separate, caller-supplied identity used to correlate
the mutation with later provider lookup. It is not inferred from
`idempotency_key`, and it is not the provider-generated `provider_request_id`
returned after a successful request. Supply the exact external lookup reference
before mutation whenever the receipt may be verified. Reusing a completed claim
with a different provider reference is an idempotency conflict.

Provider return values are intentionally not persisted. An immediate confirmed
result can expose its value, but a replay from a durable receipt may raise
`ResultValueUnavailableError` when `.value` is accessed. Persist or read back
business data in the system of record instead of relying on the ledger as a
response store.

Caller metadata must be JSON-compatible. The top-level keys `attempt_history`,
`attempt_records`, `provider`, `provider_request_id_conflict`,
`provider_request_ids`, `recovery`, `verification`, `verification_conflict`, and
`verification_observations` are reserved for Dataoad's receipt protocol and are
rejected before a claim is created.

### Local claim deduplication is not downstream idempotency

Dataoad's local claim deduplication prevents cooperating callers that share the
same ledger and key contract from deliberately starting a second operation.
Downstream idempotency is a property of the external provider, such as a
provider-enforced idempotency key or conditional write.

There is still a distributed-systems gap between committing an external side
effect and durably recording its outcome. If a ledger transition fails after
invocation, Dataoad raises `ActionPersistenceError` with the action receipt and
the in-process status/evidence it observed. The idempotency claim remains a
safety barrier, but its exact durable state may be `PENDING` or already
`COMPLETED` if the write committed before its response was lost. Inspect the
ledger; never treat the error as permission to repeat the mutation. Use
read-only recovery as described below when the receipt remains `PENDING`.
Dataoad cannot turn a non-idempotent remote API into an exactly-once
transaction. Use provider idempotency when available, in addition to Dataoad's
local claim.

Before invoking the operation, Dataoad durably increments the attempt count. If
that `start_attempt()` write succeeds but its acknowledgement is lost, the
runner reads the claim back and continues only when the receipt proves exactly
the single conditional transition it requested. It does not increment again.
If that exact transition cannot be demonstrated, the operation is not invoked
and `ActionPersistenceError` preserves the claim as a safety barrier.

## Verification semantics

A verifier is a provider-specific object or callable that receives an
`ActionReceipt` and performs read-only observation. It returns a
`VerificationResult`:

- `VerificationResult.confirmed(...)` reports positive evidence of occurrence.
- `VerificationResult.absent(...)` records a healthy absence observation but
  remains `UNKNOWN` by default. Passing `retry_safe=True` asserts the stronger
  terminal/fenced non-occurrence needed for `FAILED`.
- `VerificationResult.unknown(...)` reports missing or untrustworthy evidence.

A negative result is accepted as `FAILED` only when its health is `HEALTHY`,
`conclusive_absence=True`, and `retry_safe=True`. Other negative claims are
normalized to `UNKNOWN`.
Verifier exceptions, invalid verifier responses, stale or partial reads,
authentication failures, broken pagination, and failed positive controls must
also remain `UNKNOWN`; "could not verify" is not "did not happen."

Mutation and verification must share the receipt's durable
`provider_reference`. A decisive verifier result must echo that exact identity
as `verified_provider_reference`; missing or mismatched binding is downgraded to
`UNKNOWN`. Passing a verifier directly to `execute(...)` therefore requires
`provider_reference`. If verification will happen only through a later
`reconcile(...)` or `recover_pending(...)` call, still set the reference on the
original `execute(...)` so it is present in the durable receipt.

Pass a verifier to `execute(...)` to check immediately after ambiguous
execution, or call `runner.reconcile(action_id, verifier)` later for a completed
`UNKNOWN` receipt. Verifier correctness is application-specific: Dataoad
enforces the result contract, but cannot prove that custom evidence or a remote
system of record is truthful.

The bundled fake payment verifier sets `retry_safe=True` only after checking a
fake-provider in-flight fence as well as lookup health and absence. A production
adapter needs an equivalent provider guarantee; a healthy empty snapshot alone
is not enough.

Concurrent verifier observations are merged instead of using
first-writer-wins. Contradictory `CONFIRMED`/`FAILED` evidence makes the durable
result `UNKNOWN` and preserves both observations. Two confirmations with
different provider request IDs remain `CONFIRMED` (occurrence is still known),
but the receipt records `provider_request_id_conflict=True` and all observed IDs
because they may indicate duplicate effects. Dataoad withholds a recovered or
direct value when it cannot correlate that value to the retained provider ID.
For verifier-recovered values, two missing IDs are never considered a match: a
non-null `provider_request_id` must be present in both the verification and the
retained receipt. This keeps occurrence `CONFIRMED` while refusing an
uncorrelated optional value.

### Stranded claim recovery

Recovery is explicit and read-only; Dataoad never leases or steals an in-flight
mutation automatically.

- If an `ActionPersistenceError` leaves a `PENDING` receipt with
  `attempts >= 1`, first ensure the original worker will not issue another
  mutation, then call `await runner.recover_pending(action_id, verifier)`. A
  negative recovery can become `FAILED` only when the verifier returns
  `absent(..., retry_safe=True)` and therefore proves no late commit is
  possible. Other absence remains `UNKNOWN`.
- If a crash occurred after claim creation but before the durable attempt count
  was incremented, the receipt has `attempts == 0`. An operator may call
  `runner.release_unstarted(action_id)`. This conditionally removes only that
  uninvoked claim so the same idempotency key can be claimed again. It races
  atomically with attempt start: if invocation has begun, release is refused.

An `ActionPersistenceError.receipt` is the last safely observed receipt, not a
promise that a failed write did not commit. Re-read the ledger before choosing
between these recovery paths.

Concurrent recovery observations use the same conservative conflict merge as
normal reconciliation. `ActionPersistenceError` also exposes
`observed_status`, `observed_metadata`, and, when an immediate confirmed value
was returned, `value_available`/`observed_value`. These are diagnostic evidence,
not proof that the corresponding ledger transition committed.

## Fault harness

From a repository checkout or an unpacked source distribution, run the bundled
deterministic payment suite:

```bash
dataoad test examples/payment_timeout_after_commit.py
```

The CLI loads a Python file whose zero-argument `run_safety_suite()` function
returns a `dataoad.testing.SafetyReport`. The bundled suite covers success,
explicit pre-commit failure, replay, concurrency, key/body conflicts, faults on
both sides of commit, verifier outage, an unhealthy false-negative, a verifier
using the wrong external identity, and preservation of earlier `UNKNOWN`
evidence across later attempts.

Representative output:

```text
Timeout-after-commit comparison
  Naive retry: calls=2, charges=2, total=5000 cents
  Dataoad:    execution=UNKNOWN, verification=CONFIRMED, final=CONFIRMED, calls=1, charges=1

Action: charge_customer

Normal success                     PASS
Explicit pre-side-effect failure   PASS
Replay safety                      PASS
Concurrent duplicate safety        PASS
Same-key/body conflict             PASS
Timeout-before-commit recovery     PASS
Timeout-after-commit recovery      PASS
Unknown auto-retry protection      PASS
Verifier unavailable handling      PASS
Unhealthy false-negative handling  PASS
wrong-verification-identity        PASS
unknown-evidence-erasure           PASS

Final classification:
SAFE UNDER TESTED CONDITIONS
```

"Safe under tested conditions" describes only these deterministic scenarios.
It is not formal verification or a universal safety claim.
Each harness `SafetyCheck.passed` value must be an exact boolean; truthy values
such as the string `"false"` are rejected rather than rendered as a pass.
`SafetyReport` also rejects duck-typed or otherwise invalid check entries, so
the constructor cannot bypass that invariant.

## Architecture

```text
caller / agent
      |
      v
 ActionRunner ---- atomic claim and receipt transitions ----> Ledger
      |                                                   InMemory / SQLite
      |
      +---- mutation operation ----> external system
      |
      +---- read-only Verifier ----> system of record
```

- `ActionRunner` claims before invocation, classifies evidence, applies retry
  policy, and reconciles ambiguity.
- `ActionReceipt` is an immutable snapshot with an optimistic version. It keeps
  hashes, identifiers, bounded errors, status, and explicit JSON metadata, not
  raw request bodies or arbitrary provider responses. Structured per-attempt
  records preserve execution, verification, and retry decisions.
- `InMemoryLedger` provides thread-safe, process-local atomic claims for tests
  and ephemeral runs.
- `SQLiteLedger` provides durable local receipts and cross-process claim
  uniqueness using SQLite transactions and constraints.
- `Verifier` separates mutation execution from provider-specific read-back.
- `dataoad.testing` supplies deterministic fault injection and qualified safety
  reports; it is not a simulator of every provider failure mode.

The core has no dependency on an agent framework or LLM provider.

## Limitations and honest guarantees

Dataoad v0.1 is alpha software intended for inspection, testing, and local
integration work. In particular:

- It provides no exactly-once or production-ready guarantee.
- Correct deduplication requires all contenders to share a ledger and use a
  stable action name, idempotency key, and complete semantic request.
- `InMemoryLedger` loses state on exit. `SQLiteLedger` is local persistence, not
  a hosted, replicated, highly available coordination service.
- A hard process stop can leave a `PENDING` claim requiring explicit application
  or operator recovery. v0.1 provides conditional recovery primitives but has
  no leases, ownership service, fencing tokens, or stale-claim worker.
- The external mutation and ledger update cannot generally be one atomic
  transaction. Provider-side idempotency remains strongly recommended.
- Reconciliation is only as sound and fresh as the custom verifier and provider
  read path.
- The request hash avoids retaining a raw payload but is not encryption or an
  anonymization guarantee. Idempotency keys and explicit metadata are stored;
  do not put secrets in them. Bounded exception messages are also retained for
  diagnosis, so provider adapters should not include secrets in exception text.
- Arbitrary provider values are not durable, and there are no built-in provider
  integrations, queues, dashboards, authentication, or multi-tenancy.

## Roadmap

Likely next steps, subject to evidence from real integrations:

1. Extend explicit `PENDING` recovery with durable ownership, leases, and
   provider-aware fencing semantics.
2. Add adapter guidance and contract tests for provider idempotency and robust
   verification.
3. Add machine-readable safety reports and broader crash/concurrency fault
   cases.
4. Evaluate another transactional ledger backend when multi-host coordination
   is justified.
5. Add thin framework adapters only after the core protocol is stable.

Cloud services, a dashboard, policy languages, and broad "agent platform"
features are deliberately outside the v0.1 scope.

## Contributing

See [`CONTRIBUTING.md`](CONTRIBUTING.md). Contributions should preserve the
central invariant: ambiguous side effects remain `UNKNOWN` unless trustworthy
evidence proves otherwise.

## License

Licensed under the Apache License 2.0. See [`LICENSE`](LICENSE).
