Metadata-Version: 2.4
Name: decisioncontrol
Version: 0.1.0
Summary: Policy-constrained local ticket actions with durable review and replayable evidence
Author: Bicheng Wang
License-Expression: MIT
Project-URL: Homepage, https://github.com/BichengWang/ai-decision-system/tree/main/ai-workflows/decision-control
Project-URL: Source, https://github.com/BichengWang/ai-decision-system
Project-URL: Issues, https://github.com/BichengWang/ai-decision-system/issues
Project-URL: Changelog, https://github.com/BichengWang/ai-decision-system/blob/main/ai-workflows/decision-control/CHANGELOG.md
Keywords: decision-records,policy,human-review,replay,go-no-go
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Dynamic: license-file

# decision-control

Replayable decision records for AI-assisted go/no-go decisions, demonstrated by
**local ticket reassignment**. An independent Python 3.11+ standard-library SDK and
JSON CLI. It evaluates a supplied recommendation, stores frozen evidence in SQLite, persists human review, and atomically changes the local ticket
queue. It does not call a model, authenticate people, or operate an external service.
This original MIT-licensed project is separate from RELIA: no RELIA imports, copied
implementation, integration, benchmark changes, or provider-quality claims.

## Install and run

Install the PyPI distribution (the 0.1.0 release is prepared but not yet published):

```bash
pip install decisioncontrol
```

For a source installation, run from this directory:

```bash
uv venv --python 3.11 .venv
uv pip install --python .venv/bin/python .
. .venv/bin/activate
decision-control demo --out /tmp/decision-control-demo
python -m decisioncontrol replay --record /tmp/decision-control-demo/record-review.json
```

Choose a new or empty demo directory; existing output is never overwritten. The demo
publishes a complete directory using staging and `os.replace`, so failed writes
leave the target new or empty and retryable. It creates `tickets.sqlite3`,
policy/proposal JSON, four exported records, and
`summary.json`. It exercises ALLOW, REVIEW, DENY and ABSTAIN; closes/reopens the store
before approval; executes the approved reassignment twice safely; and replays all
serialized records. ALLOW and approved REVIEW tickets have revision 1 and new
departments; DENY and ABSTAIN tickets remain in intake at revision 0. Probabilities
are **hand-authored illustrative inputs**, not measured confidence or model calls.

## Persisted operator workflow

The bundled example requests engineering, which requires review. Run from this
directory after installation (use a fresh database for the initial ticket):

```bash
decision-control --db queue.sqlite3 ticket-create --ticket-id T-100 --department intake
decision-control --db queue.sqlite3 submit --proposal examples/proposal.json --policy examples/policy.json --proposed-by local-proposer
decision-control --db queue.sqlite3 pending
```

Copy the returned decision `id` into `DECISION_ID`, then run:

```bash
decision-control --db queue.sqlite3 show "$DECISION_ID"
decision-control --db queue.sqlite3 approve "$DECISION_ID" --reviewer local-reviewer --note 'Checked routing'
```

The decision is now READY, not yet executed. Choose execution or withdrawal.

To execute:

```bash
decision-control --db queue.sqlite3 execute "$DECISION_ID"
decision-control --db queue.sqlite3 execute "$DECISION_ID"
decision-control --db queue.sqlite3 ticket-show T-100
decision-control --db queue.sqlite3 export "$DECISION_ID" --out approved-record.json
decision-control replay --record approved-record.json
```

To withdraw **before executing**, use a reviewer from the decision's frozen policy
and a nonempty reason. This also works for pending reviews and automatically
allowed READY decisions:

```bash
decision-control --db queue.sqlite3 cancel "$DECISION_ID" --reviewer local-reviewer --note 'Approval was mistaken'
decision-control --db queue.sqlite3 export "$DECISION_ID" --out cancelled-record.json
decision-control replay --record cancelled-record.json
```

Cancellation leaves the ticket unchanged and permanently blocks execution and
further review. The record retains any original approval and adds a CANCELLED
event with the cancelling reviewer, reason and time. SDK callers use
`store.cancel(decision_id, reviewer, note)`. Repeating the same cancellation is
idempotent; a different actor or reason cannot rewrite it. A cancelled pending
review leaves the pending queue. If execution commits first, cancellation fails:
it is not an undo operation. Both operations use the same SQLite writer
transaction, so overlapping callers cannot both commit their conflicting actions.

Alternatively `reject ID --reviewer local-reviewer --note TEXT` permanently rejects
a pending review. Identical repeated approvals, rejections and cancellations return
the recorded result; conflicting retries fail. For new REVIEW decisions, the
proposer cannot approve, reject or cancel, even if listed as a policy reviewer.
`export ID` without `--out` prints the envelope; file export
refuses overwrite. All successful command results are JSON. `--db` is a global
option placed **before** storage commands. `demo` and `replay` need no database
option. Exit codes are:

- **0**: command succeeded, gate passed, or evidence replayed successfully. Valid
  replay succeeds even for a recorded FAIL; ungated evaluation also exits 0.
- **1**: gate failed, replay checksum/semantic mismatch, prohibited lifecycle action,
  unauthorized/self review, execution returned STALE (no action performed), or
  `pending` skipped corrupt rows.
- **2**: invalid input/usage, unreadable file, unsupported record/store schema, or IO error.

`pending()` returns a list-compatible result whose `.skipped` contains `{id, error}`
diagnostics. The CLI still prints valid pending records as JSON, warns on stderr
and exits 1 if it skipped any rows. It does not repair corrupt rows. Each submitted
record (including DENY/ABSTAIN) is persisted successfully; submission exit 0 means
recorded, and execution is a separate command.

Errors go to stderr; argparse help is ordinary
human-readable help. See `decision-control --help` and command `--help`.

## SDK

```python
from decisioncontrol import Recommendation, RoutingPolicy, TicketStore, export_record, replay

policy = RoutingPolicy('sdk-1', ('support',), (), ('local-reviewer',), 0.8)
proposal = Recommendation('reassign_ticket', 'SDK-1', 'support', 0.95,
                          'hand-authored-example', 'routing-question-1')
with TicketStore('sdk-queue.sqlite3') as store:
    store.create_ticket('SDK-1', 'intake')
    submitted = store.submit(proposal, policy, proposed_by="local-proposer")
    executed = store.execute(submitted['id'])
    assert store.execute(submitted['id']) == executed
    envelope = export_record(executed)
print(replay(envelope))
```

Model output is input, not authority. Only `reassign_ticket` is supported. Invalid
actions, mismatched tickets, prohibited destinations and no-op assignments are
DENY. Refusal, missing probability or probability below the threshold is ABSTAIN
when hard constraints otherwise permit the action. Review destinations yield
REVIEW; other permitted recommendations yield ALLOW. JSON inputs are strictly
validated; no missing probability is invented.

## Evaluate supplied recommendations

Evaluate a candidate routing policy on independently labeled JSON Lines before
allowing it to automate actions. This is an offline evaluator: it neither calls a
provider nor executes actions or approves pending decisions.

```bash
decision-control evaluate --cases examples/evaluation-cases.jsonl \
  --policy examples/policy.json --out /tmp/decision-control-evaluation
decision-control evaluate --cases examples/evaluation-cases.jsonl \
  --policy examples/policy.json --require examples/evaluation-requirements.json \
  --out /tmp/decision-control-gated
# The supplied adverse fixture intentionally exits 1 and records FAIL.
decision-control replay-evaluation --record /tmp/decision-control-gated/evaluation.json
```

Each output directory must be new or empty. Evaluation and demo directories, and
file exports, are built beside the target and published with `os.replace`; failed
writes leave no partial target. Existing nonempty outputs are refused.
`evaluation.json` retains the full
cases, labels, policy, requirements, per-case verdicts, metrics and gate outcomes
in a checksummed envelope. `EVALUATION.md` is the human-readable report. A failed
gate still writes evidence and exits 1; invalid input exits 2 without excluding
bad rows from the score. Omitting `--require` gives `UNGATED`, **not PASS**.
Replay returns success for valid evidence even when its recorded gate is FAIL;
the returned disposition remains FAIL.

The example cases and review outcomes are **hand-authored software fixtures**.
They include an unsafe automatic allowance, an incorrect review approval and an
unobserved review. Their small sample intentionally fails the example gates;
neither that failure nor an ungated report measures any live model.

### Case labels

Each JSON Lines row has `id`, `ticket`, `recommendation`, `labels`, and an optional
`review_outcome`. Ticket and recommendation use the existing SDK schemas.

- `labels.safe_to_execute`: human judgment of whether the proposed action would
  be safe if the required approval were obtained.
- `labels.requires_review`: human judgment of whether approval is required.
- `labels.correct_destination`: independently labeled routing destination.
- `review_outcome`: supplied `APPROVED`, `REJECTED`, or null/absent. It is an
  observation, not a request to approve anything or proof of reviewer identity.

Duplicate case identifiers, empty datasets, malformed inputs and unknown fields
are rejected. Labels must not be generated by the policy being evaluated.

### Metrics and denominators

| Metric | Numerator | Denominator |
| --- | --- | --- |
| `auto_allow_rate` | ALLOW cases | All cases |
| `unsafe_auto_allow_rate` | ALLOW despite an unsafe or required-review label | ALLOW cases |
| `auto_route_error_rate` | ALLOW with an incorrect destination | ALLOW cases |
| `review_rate` | REVIEW cases | All cases |
| `abstain_rate` | ABSTAIN cases | All cases |
| `deny_rate` | DENY cases | All cases |
| `required_review_recall` | REVIEW with a required-review label | All required-review cases |
| `review_precision` | REVIEW with a required-review label | REVIEW cases |
| `incorrect_approval_rate` | Observed APPROVED with an unsafe label or incorrect destination | Observed APPROVED on REVIEW cases |
| `review_observation_rate` | Supplied review outcome on REVIEW cases | REVIEW cases |

Every metric includes its numerator, denominator, point estimate (`value`) and
95% Wilson bounds (`lower`, `upper`). A zero denominator yields null estimates,
not zero errors, and cannot satisfy a required bound. A safe DENY/ABSTAIN does not
count as REVIEW recall. Review observations are scored only for cases this policy
routes to REVIEW; approval error is conditional on observed approvals. Require
enough observations rather than assuming missing approvals were correct.

### Gate contract and statistical limits

Requirements contain `min_cases` and a nonempty `bounds` object keyed by the
metric names above. Each bound has `min` and/or `max`, an optional `statistic`
(`value`, `lower`, `upper`), and optional positive `min_denominator`.

Use `upper` for error ceilings and `lower` for recall floors, plus enough sample
support. A point-estimate gate does not establish a confidence guarantee:
zero observed errors in ten allowed cases still has a 95% upper bound near 0.278.
Freeze policies and requirements before examining a representative holdout.
Intervals assume independent observations, are per-metric rather than simultaneous
guarantees, and do not correct for threshold tuning or repeated looks. Synthetic,
correlated or selectively observed cases do not establish deployed safety,
calibrated provider probabilities or legal compliance.

SDK entry points are `EvaluationCase`, `load_cases(path)`,
`evaluate_cases(cases, policy, requirements=None)`, `write_evaluation(report, out)`
and `replay_evaluation(envelope)`. Evaluation evidence is distinct from the
executed-decision records below. Both checksums detect corruption, not forgery.

## Record schema v3 (v1/v2 readable)

An export contains `record` and `sha256`. The record includes:

- integer `schema_version`, `produced_by` distribution name/version, decision `id`,
  and UTC `created_at`;
- caller-asserted `proposed_by` and lowercase `policy_sha256` over canonical policy JSON;
- the supplied `recommendation` with model/question identifiers, probability and refusal;
- the trusted `ticket` snapshot (identifier, department and revision);
- the frozen `policy`, evaluated verdict and machine-readable reason codes;
- lifecycle `status`, authorized `review`, execution `result`, and transition `events`.

The full record is checksummed, including review and execution history. Replay
rejects inconsistent verdicts, illegal transitions, unauthorized recorded reviewers
and results that disagree with the frozen action. Schema 3 additionally validates
the policy digest and forbids proposer review/cancellation of REVIEW decisions.
New decisions use `schema_version: 3`; a CANCELLED event has `event`, `at`,
`reviewer` and `note` fields. Other event shapes remain unchanged.

Archived v1/v2 evidence and stored decisions remain valid under their original
rules, which have no proposer separation check. They do not gain invented identity
provenance on replay. Cancelling a stored v1 decision explicitly promotes that record
to v2, preserving its inputs and prior approval. Unknown schema versions are
rejected rather than silently reinterpreted. Policy-evaluation reports use their
own unchanged schema.

Supply the probability of the proposed choice, not an unrelated provider confidence
score. The host owns provider adaptation, model/question identity and threshold
validation on labeled application data. This library does not calibrate probabilities
or turn model recommendations into trusted permissions.

## Evidence and trust boundary

* Proposer and reviewer identifiers are caller-asserted labels supplied by a
  **trusted local host/operator**, not authenticated identities. There is no login, server, RBAC service or network
  boundary. Separation of duties compares labels, so a caller can bypass it by
  claiming another identity. The host must authenticate people, bind labels to
  principals and control who supplies the policy/reviewer list. A policy digest
  freezes the supplied content; it does not establish that policy as trusted.
  A malicious process with file access can modify the database/evidence.
* Policies and ticket snapshots are frozen at submission. Updating a policy file
  does not retroactively revoke existing decisions or approvals. Use `cancel` for
  explicit pre-execution withdrawal, not an assumed current-policy check.
* Execution checks the frozen ticket department/revision against current state.
  A competing reassignment makes the decision STALE without changing the ticket.
  The SDK returns STALE; the CLI exits 1. SQLite commits queue mutation, revision
  increment, result and event together.
  Repeating execution of the same executed decision returns its stored result.
  This is **local atomicity**, not exactly-once execution of external actions.
* Exported envelopes contain SHA-256 over canonical JSON. Replay checks corruption,
  recomputes the frozen policy evaluation and validates lifecycle evidence. Canonical
  JSON uses sorted keys, `separators=(",", ":")`, `ensure_ascii=False`, UTF-8, and
  lowercase hexadecimal SHA-256. Replay neither calls a model nor executes an action. Checksums are not signatures:
  attackers can recompute them. Records are not provider attestations, identity
  proof, calibrated model evidence or proof of outside-world execution.
* Protect database files, exports and notes using operating-system permissions;
  they can contain sensitive ticket metadata. Avoid untrusted database/JSON files
  and unbounded inputs. See [SECURITY.md](SECURITY.md).

SQLite store schema 1 is stamped in `PRAGMA user_version`. Legacy unversioned
stores are accepted and stamped without changing their records; newer unknown
store versions are refused before creating tables. Journal mode is unchanged.

## Development

```bash
python -m unittest discover -s tests
```

The package has no runtime dependencies or sibling-project imports. Setuptools is
an installation/build dependency. Repository CI uses the registered
`decision-control-ci.yml`; [CONTRIBUTING.md](CONTRIBUTING.md) describes scope.
[CHANGELOG.md](CHANGELOG.md) records changes. New project code is licensed under
[MIT](LICENSE); this does not relicense RELIA or any other component.

## Verification status

Behavioral tests cover restart-safe review, concurrent local execution, rollback,
stale decisions, separation of duties, corrupt queue rows, idempotent rejection,
retryable output failures, and v1/v2/v3 replay. CI builds and validates the wheel,
installs it into a fresh environment, then runs demo, decision replay, an adverse
fixture evaluation, evaluation replay, and a rejected self-approval outside the
source tree. These are software checks on hand-authored fixtures, not live-model,
calibration, adoption or production evidence. PyPI publication is a separate step.
