Metadata-Version: 2.5
Name: testseal
Version: 1.0.1
Summary: Deterministic test-suite integrity checks for Python and pytest diffs
Project-URL: Homepage, https://testseal-integrity.vercel.app
Project-URL: Repository, https://github.com/satwiksps/testseal.git
Project-URL: Issues, https://github.com/satwiksps/testseal/issues
Project-URL: Changelog, https://github.com/satwiksps/testseal/blob/main/CHANGELOG.md
Project-URL: Documentation, https://testseal.readthedocs.io/
Author: TestSeal contributors
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: ai-agents,ci,code-review,pytest,static-analysis,test-integrity,testing,unittest
Classifier: Development Status :: 5 - Production/Stable
Classifier: Environment :: Console
Classifier: Framework :: Pytest
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Software Development :: Quality Assurance
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.11
Provides-Extra: dev
Requires-Dist: build>=1.2; extra == 'dev'
Requires-Dist: pre-commit>=4.0; extra == 'dev'
Requires-Dist: pytest-cov>=5.0; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff<1,>=0.11; extra == 'dev'
Requires-Dist: twine>=6.0; extra == 'dev'
Description-Content-Type: text/markdown

<h1 align="center">TestSeal</h1>

<p align="center">
  <img src="https://raw.githubusercontent.com/satwiksps/testseal/main/docs/assets/testseal-banner.svg" alt="TestSeal">
</p>

<p align="center"><strong>Deterministic test-integrity checks for Python and pytest diffs.</strong></p>

<p align="center">
  <a href="https://testseal-integrity.vercel.app">Website</a> |
  <a href="https://testseal.readthedocs.io/">Documentation</a> |
  <a href="https://testseal.readthedocs.io/en/latest/rules/">Rule reference</a> |
  <a href="https://testseal.readthedocs.io/en/latest/architecture/">Architecture</a> |
  <a href="https://github.com/satwiksps/testseal/blob/main/CONTRIBUTING.md">Contributing</a>
</p>

<p align="center">
  <a href="https://github.com/satwiksps/testseal/actions/workflows/ci.yml"><img src="https://github.com/satwiksps/testseal/actions/workflows/ci.yml/badge.svg?branch=main" alt="CI"></a>
  <a href="https://pypi.org/project/testseal/"><img src="https://img.shields.io/pypi/v/testseal?logo=pypi&amp;logoColor=white" alt="PyPI"></a>
  <a href="https://codecov.io/gh/satwiksps/testseal"><img src="https://codecov.io/gh/satwiksps/testseal/graph/badge.svg?branch=main" alt="Codecov"></a>
  <a href="https://www.python.org/downloads/"><img src="https://img.shields.io/badge/python-3.11%2B-3776AB?logo=python&amp;logoColor=white" alt="Python 3.11+"></a>
  <a href="https://github.com/satwiksps/testseal/blob/main/LICENSE"><img src="https://img.shields.io/github/license/satwiksps/testseal" alt="License"></a>
</p>

TestSeal compares tests before and after a change and reports concrete weakening
signals: removed assertions, newly disabled tests, weaker comparisons, wider
tolerances, swallowed exceptions, snapshot regeneration, and suspicious mocks.
It runs locally, needs no model or API key, and never imports or executes the
repository it scans.

```diff
- assert total == Decimal("19.99")
+ assert total
```

After [installing TestSeal](#quick-start), run the built-in example from any
directory. It does not read Git, configuration, or the network:

```bash
testseal demo
```

```text
[HIGH] TS003 tests/test_totals.py:10:1 - Assertion weakened
  A specific equality assertion was replaced by a truthy/non-null check
  Evidence: assert total == Decimal("19.99")  ->  assert total
  Fingerprint: 3c056c0da89673cd1a42eacc
  Fix: Assert the specific expected value, type, relationship, or exception.

TestSeal: 1 finding(s) in 1 changed file(s) (high 1, medium 0, low 0).
```

## Why TestSeal

A green test suite is weak evidence when the same change made the suite easier
to pass. Ordinary linters accept both sides of the example because both are
valid Python. General code reviewers may notice the downgrade, but their output
is probabilistic. TestSeal provides a narrow, reproducible signal dedicated to
how the tests themselves changed.

- Identical input and configuration produce identical output.
- The Python package has no runtime dependencies, model calls, telemetry, or accounts.
- Git-backed scans compare complete before/after syntax when both blobs are available.
- Scans read source and Git data without importing the target project.
- Findings are advisory unless a failure threshold is configured.
- Reports are available as text, versioned JSON, and SARIF 2.1.0; integrations are
  provided for pre-commit and GitHub Actions.

## Quick start

TestSeal requires Python 3.11 or newer and Git for repository-backed scans:

```bash
python -m pip install testseal
testseal demo
```

For an isolated CLI installation:

```bash
uv tool install testseal
# or: pipx install testseal
```

Then run it in a Git repository:

```bash
testseal scan
```

The default scan is advisory. Add `--fail-on high` when high-severity findings
should return exit code `1`.

## Scan modes

| Goal | Command |
| --- | --- |
| Check working-tree and untracked changes | `testseal scan` |
| Check staged changes | `testseal scan --staged` |
| Compare a branch with its base | `testseal scan --base origin/main --head HEAD` |
| Read a unified diff | `testseal scan --diff changes.patch` |
| Block on high-severity findings | `testseal scan --fail-on high` |

Exit codes are `0` for a completed advisory scan, `1` when the selected finding
threshold is met, and `2` for invalid configuration or an incomplete blocking
scan.

### Output

Text is the default. JSON and SARIF can be printed or written atomically:

```bash
testseal scan --base origin/main --format json --output testseal-report.json
testseal scan --base origin/main --format sarif --output testseal-report.sarif
```

Run `testseal scan --help` to list the available CLI options.

## Configure policy

TestSeal discovers `testseal.toml` first, then `[tool.testseal]` in
`pyproject.toml`. An explicit `--config PATH` takes precedence.

```toml
[testseal]
fail_on = "high"
test_patterns = ["test_*.py", "*_test.py", "tests/**/*.py"]
source_roots = ["src"]
disabled_rules = ["TS008"]

# Copy a fingerprint from text, JSON, SARIF, or the Action output after review.
ignore_fingerprints = ["3c056c0da89673cd1a42eacc"]

[testseal.rules.TS006]
severity = "low"
```

Configuration is strict: unknown keys, rule IDs, severities, and malformed
fingerprints fail with exit code `2` instead of silently weakening policy.
See the repository's
[default](https://github.com/satwiksps/testseal/blob/main/testseal.toml),
[strict](https://github.com/satwiksps/testseal/blob/main/examples/configs/strict.toml),
and [monorepo](https://github.com/satwiksps/testseal/blob/main/examples/configs/monorepo.toml)
examples.

## Pre-commit

The hook installs TestSeal in its own environment and scans the staged diff:

```yaml
repos:
  - repo: https://github.com/satwiksps/testseal
    rev: v1.0.1
    hooks:
      - id: testseal
        args: ["--fail-on", "high"] # omit to remain advisory
```

## GitHub Actions

The Action installs the Python core bundled in the same release, derives pull
request refs from the event payload, annotates changed lines, and exposes a
normalized JSON result.

```yaml
name: Test integrity

on: [pull_request]

permissions:
  contents: read

jobs:
  testseal:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
        with:
          fetch-depth: 0
      - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
        with:
          python-version: "3.12"
      - id: testseal
        uses: satwiksps/testseal@e6bba7e933c37afc34e2836ac2b1baee7542bfe5 # v1.0.0
        with:
          fail-on: high
```

Omit `fail-on` to honor repository configuration. The default `install: true`
uses only the source bundled with that Action commit. Set
`install: false` only when the same TestSeal version is already installed in the
selected Python environment.

For accepted CLI reports, Action outputs include `finding-count`, severity counts,
`files-scanned`, `suppressed-count`, `outcome`, and the normalized `result` JSON.
Input, installation, process, and rejected-report failures set only `outcome`.

## Rules

| Rule | Signal | Severity | Confidence |
| --- | --- | --- | --- |
| `TS001` | An assertion was removed from a test | High | High |
| `TS002` | A pytest or unittest skip/xfail was added | High | High |
| `TS003` | An assertion was replaced with a weaker form | High | High |
| `TS004` | A comparison tolerance was widened | High | High |
| `TS005` | A broad exception is now swallowed | High | High |
| `TS006` | Snapshot update or regeneration behavior was added | Low | Low |
| `TS007` | The apparent subject under test is now mocked | Medium | Medium |
| `TS008` | Source and a configured guarding test changed together | Low | Low |

The table is a summary. The
[rule reference](https://testseal.readthedocs.io/en/latest/rules/)
defines supported syntax, intentional non-findings, and precision limits.
Context-heavy rules should be baselined before enabling a blocking threshold.

## Trust boundary and limitations

TestSeal invokes Git to obtain refs, diffs, and blobs, then parses Python source
with the running interpreter. It does not run tests, import changed modules, or
execute hooks from the target repository. Use `pull_request`, read-only
permissions, and no repository secrets when scanning contributions from forks.

The analyzer reports specific transformations; it does not decide whether an
author is honest, prove that tests are complete, or replace code review,
coverage, linters, type checkers, and security analysis. A finding can describe
a legitimate refactor, so blocking is explicit and reviewed exceptions use
stable fingerprints rather than hidden heuristics.

The analyzer currently targets Python test files. It understands common pytest,
unittest, and `unittest.mock` patterns. The TypeScript code in this repository
implements the GitHub Action and website; it does not analyze JavaScript or
TypeScript test suites.

Read the full [architecture and trust model](https://testseal.readthedocs.io/en/latest/architecture/)
and [security policy](https://github.com/satwiksps/testseal/blob/main/SECURITY.md).

## Development

```bash
git clone https://github.com/satwiksps/testseal.git
cd testseal
python -m venv .venv
# POSIX: source .venv/bin/activate
# Windows PowerShell: .venv\Scripts\Activate.ps1
python -m pip install -e ".[dev]"
python -m pytest
```

The TypeScript Action lives in `packages/action` and the Next.js/Tailwind site
in `site`. See [CONTRIBUTING.md](https://github.com/satwiksps/testseal/blob/main/CONTRIBUTING.md)
for the complete verification commands and rule-change requirements.

## Support

Use [GitHub Issues](https://github.com/satwiksps/testseal/issues) for reproducible
bugs and rule false positives. Report vulnerabilities through
[GitHub private vulnerability reporting](https://github.com/satwiksps/testseal/security/advisories/new).

## License

TestSeal is licensed under the [Apache License 2.0](https://github.com/satwiksps/testseal/blob/main/LICENSE).
