Metadata-Version: 2.4
Name: silentfail-sdk
Version: 0.1.1
Summary: Verify that your automation actually did the work, inside your own process.
License: MIT
Project-URL: Homepage, https://silentfail.app
Classifier: License :: OSI Approved :: MIT License
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Dynamic: license-file

# silentfail

Verify that your automation actually did the work, inside your own process.

A deadline catches a workflow that stops running. This catches one that runs,
reports success, and does not do the work.

You describe the job in the dashboard, read the checks it compiles into, and
approve them. From then on this package checks each run against those checks
next to your own data. **A check that compares values sends us the verdict and
nothing else.** The values it was computed from never leave your machine, and
there is no field on the wire format that one would fit in.

One kind of check does transmit. A rule may include a judged check, for a
question that comparing values cannot settle: whether a summary reflects the
email it came from, whether a reply answers what was asked. A judged check names
the fields it needs, you see that list before you approve the rule, and only the
fields it names are sent, to a maximum of six. A rule with no judged check sends
nothing but the verdict.

Python 3.9 or newer. No dependencies, deliberately: this installs into an
environment that is already pinned to something.

## Use

```python
import os
from silentfail import Supervisor, Claim

supervisor = Supervisor.start(
    api_key=os.environ["SILENTFAIL_API_KEY"],
    monitor_id="cmu0480ko0007ssi0f1zz7d60",
)

# ... your job runs, and files invoice INV-4471 ...

supervisor.check(
    run_id=job_id,
    claim=Claim(status="success", reference="INV-4471"),
    evidence=lambda: ledger.find(invoice_no="INV-4471"),
)

supervisor.flush()   # for a Lambda, a cron job, anything that exits when done
```

`claim` is what your run says it did. `evidence` is what an independent source
says happened, fetched by your code, from your systems.

## The two rules

**`start()` raises. `check()` never does.**

`start()` runs once, at startup, with a developer watching. A mistyped key or a
monitor with no approved rule is a deploy-time problem, and finding out at three
in the morning by noticing that nothing has been verified for a month is not
acceptable, so it fails immediately and says what to fix.

`check()` runs in production at the end of a real job. It has no raising path,
for any input, any network condition, or anything that has gone wrong on our
side. A monitoring tool that takes down the thing it monitors is worse than no
monitoring tool.

An account problem is not a setup mistake. An unpaid invoice or a paused
subscription is decided on our side and can happen months after this was
deployed, so it logs loudly and your process carries on running unverified. Our
billing system does not get to stop your production job.

## It does not block

The rule is fetched once at startup and cached. Verdicts go out on a background
daemon thread, so `check()` returns as soon as the local evaluation is done and
a daemon thread never delays your interpreter exiting. Call `flush()` when you
want to wait for them.

Your `evidence` callable is bounded by a timeout (10 seconds by default). Python
cannot kill the thread still waiting on a slow query, so that thread is left to
finish on its own, but **your** call returns at the deadline. Timing out is
reported honestly as "could not check".

## Three outcomes, not two

Every check is `passed`, `failed`, or `could_not_check`, and the third is the
important one. A check that did not run is never reported as a pass.

## Agreement with the Node package

The evaluator here is a reimplementation, not a binding. The two are held
together by `supervisor-conformance/`, a suite of rule, input and expected
verdict fixtures that both packages run, written from the format's documented
semantics rather than recorded from either implementation.

Four Python behaviours would silently disagree with JavaScript without the
fixtures catching them, and all four have cases:

- `0 == False` and `1 == True` are true here and false there. Equality compares
  the type first.
- `isinstance(True, int)` is true, so a boolean would satisfy every numeric
  comparison. Numeric tests exclude `bool` explicitly.
- JSON `1` parses to `int` and `1.0` to `float`, where JavaScript has one number
  type. Whole-number tests ask about the value, not the type.
- Dictionary lookup has no prototype chain, so unlike the JavaScript version
  there is nothing here that needs guarding against a path resolving to
  `constructor`.

Run them:

```bash
cd packages/silentfail-python && python -m unittest discover -s tests -t .
```
