Metadata-Version: 2.5
Name: testpulse
Version: 0.1.0
Summary: Time your tests, optimize your outcomes.
Project-URL: Homepage, https://github.com/MaruthiCharan2403/testpulse
Project-URL: Documentation, https://github.com/MaruthiCharan2403/testpulse#readme
Project-URL: Repository, https://github.com/MaruthiCharan2403/testpulse
Project-URL: Issues, https://github.com/MaruthiCharan2403/testpulse/issues
Author: TestPulse contributors
License: MIT
License-File: LICENSE
Keywords: cli,developer-productivity,pytest,testing
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Testing
Classifier: Typing :: Typed
Requires-Python: >=3.11
Description-Content-Type: text/markdown

# TestPulse

**Time your tests, optimize your outcomes.**

Turn your current git diff into a single, evidence-based pytest command by learning from locally recorded test timings, failures, and which tests historically correlate with your changes; quantify flakiness and produce explicit risk guidance for skipping tests.

TestPulse is a **Python library and CLI** (`testtime`). It stores history in a **local SQLite database** (stdlib only—no runtime dependencies). There is **no web application** and **no HTTP API server**; you use pytest, the terminal, and optional library imports.

## Features

### Local test recording (pytest plugin)

**What it does:** Records each test’s nodeid, duration, pass/fail/skip/xfail outcome, and hashes of captured stdout/stderr into SQLite. Associates the run with the current git diff paths and updates a **change↔test co-occurrence** graph for later recommendations.

**How to use:**

```bash
export PYPI_PACKAGE_NAME=testpulse
pytest --testpulse-record
pytest --testpulse-record --testpulse-ci          # quiet CI mode
pytest --testpulse-record --testpulse-store /path/to/store.sqlite3
```

Default store: `.testpulse/store.sqlite3` (override with `TESTPULSE_STORE`).

---

### Recommended pytest command (`testtime run`)

**What it does:** Reads your git diff (or a commit range), maps changed files to pytest targets via **static heuristics** and **learned co-occurrence**, weights candidates by **expected reliability** from flakiness history, and prints one exact `pytest ...` line.

**How to use:**

```bash
testtime run
testtime run --commit-range main..HEAD
testtime run --min-reliability 0.8 --store .testpulse/store.sqlite3
```

Copy the printed command and run it in your shell.

---

### Flakiness estimates (Bayesian smoothing)

**What it does:** Per-test **flaky probability** (inconsistent pass/fail history), **confidence band** (low / medium / high by sample size), and **expected reliability** for ranking and skip-risk logic.

**How to use (library):**

```python
from testpulse import Store, flakiness_estimates, flakiness_for_nodeid
from testpulse.paths import default_store_path

store = Store.open(default_store_path())
try:
    for est in flakiness_estimates(store):
        print(est.nodeid, est.flaky_probability, est.expected_reliability)
finally:
    store.close()
```

Also surfaced in `testtime report` and in recommendation / risk scoring.

---

### Skip and regret risk

**What it does:** For tests **not** included in the recommendation, computes a **regret risk** score (likelihood of missing a real failure) and an aggregate **risk level**: `low`, `medium`, `high`, or `critical`.

**How to use (library):**

```python
from testpulse import compute_skip_risk_report

report = compute_skip_risk_report(["src/mod.py"], store)
print(report.aggregate_level, report.max_regret_risk)
for item in report.excluded:
    print(item.target, item.regret_risk)
```

Shown in `testtime report` (menu **6**) and `--no-interactive` skip-risk section.

---

### Regret hits

**What it does:** Across **consecutive recorded runs**, finds tests that **passed** in an earlier run but **failed** in a later run when diffs link through shared changed paths or learned co-occurrence—tests that “should have caught” a later breakage.

**How to use (library):**

```python
from testpulse import regret_hits

for hit in regret_hits(store):
    print(hit.nodeid, hit.prior_run_id, hit.later_run_id, hit.linking_path)
```

Listed in `testtime report` (menu **5**).

---

### Interactive report (`testtime report`)

**What it does:** Terminal menu to explore:

1. Slowest tests (average duration)  
2. Most flaky tests  
3. Most change-relevant tests (for current diff)  
4. Top drivers of today’s recommendation  
5. Regret hits  
6. Skip / regret risk summary  

**Filter:** press **`f`** to filter by pytest **marker** or **class** name (parsed from stored markers and nodeids).

**How to use:**

```bash
testtime report
testtime report --no-interactive    # drivers + skip risk only, then exit
testtime report --commit-range HEAD~5..HEAD
```

---

### Explanations with local citations (`testtime explain`)

**What it does:** For each included/excluded candidate, explains **why** and which **changed files** relate, with **citations** pointing at rows in your SQLite history (heuristic maps, co-occurrence links, outcome counts)—no external datasets.

**Optional LLM:** With `--llm` and your own API key, adds a short narrative constrained to those citations only.

**How to use:**

```bash
testtime explain
testtime explain --llm
```

Exit code **3** if `--llm` is requested but no API key is configured.

---

### CI stats export (de-identified)

**What it does:** Exports **aggregated** timing and outcome stats (run counts, outcome totals, duration percentiles, flaky buckets). **No** test logs, **no** raw nodeids in the JSON (only counts and hashed identifiers internally). Optional HTTPS upload when URL and token env vars are set.

**How to use:**

```bash
testtime ci-export --output testpulse-ci-stats.json
testtime ci-export --upload
```

GitHub Actions workflow `.github/workflows/testpulse-stats.yml` records (optional), exports JSON, and uploads an artifact.

---

### Brand assets

Static logo and favicon for documentation and packaging: `frontend/public/logo.svg`, `frontend/public/favicon.svg` (primary `#003366`, accent `#FFCC00`). Not a runnable UI.

## Pages

TestPulse does **not** ship a browser frontend or routes. User-facing surfaces are:

| Surface | What you see |
|---------|----------------|
| **Terminal — `testtime report`** | Menu of report sections; optional marker/class filter prompts |
| **Terminal — other commands** | Single-shot stdout (pytest command, explanations, CI JSON path) |
| **Pytest session** | Optional line `testpulse: recorded N test(s)` when recording (hidden in `--testpulse-ci`) |

There are no `/login`, settings, or dashboard URLs—only the CLI and library APIs below.

## API

There is **no REST backend**. The stable **Python public API** is exported from `testpulse` (defined in `testpulse.api`). Open a `Store` unless noted.

### Package metadata

| Symbol | Purpose |
|--------|---------|
| `VERSION`, `tagline` | Version string and product tagline |
| `distribution_name()` | Read `PYPI_PACKAGE_NAME` |
| `import_package_name()` | Distribution name with `-` → `_` for imports |

### Recording and storage

| Symbol | Purpose |
|--------|---------|
| `Store.open(path)` / `Store.connect_memory()` | SQLite store (`testpulse.store`) |
| Pytest plugin | Entry point `testpulse` — flags `--testpulse-record`, `--testpulse-ci`, `--testpulse-store` |

### Recommendations

| Symbol | Purpose |
|--------|---------|
| `recommend_pytest_command(changed_paths, store, min_reliability=0.0)` | One `pytest ...` shell line |

### Flakiness

| Symbol | Purpose |
|--------|---------|
| `flakiness_estimates(store)` | All tests with `FlakinessEstimate` |
| `flakiness_for_nodeid(store, nodeid)` | Single test or `None` |
| `flaky_probability(passes, fails)` | Smoothed flaky score |
| `beta_smoothed_pass_rate(passes, fails)` | Posterior mean pass rate |
| `confidence_band(passes, fails)` | `low` / `medium` / `high` |
| `estimate_from_counts(OutcomeCounts)` | Build `FlakinessEstimate` |

### Skip / regret risk

| Symbol | Purpose |
|--------|---------|
| `compute_skip_risk_report(changed_paths, store, …)` | `SkipRiskReport` with excluded tests |
| `regret_risk(relevance, fail_probability, max_relevance=…)` | Single regret score |
| `aggregate_risk_level(excluded)` | Overall `RiskLevel` |

### Regret hits

| Symbol | Purpose |
|--------|---------|
| `regret_hits(store)` | List of `RegretHit` across consecutive runs |

### Reports

| Symbol | Purpose |
|--------|---------|
| `build_report_snapshot(changed_paths, store, …)` | `ReportSnapshot` for all sections |
| `run_interactive_report(changed_paths, store, …)` | Interactive menu |
| `render_report_view(snapshot, view, filter, …)` | Print one section (`slowest`, `flaky`, `relevant`, `drivers`, `regret`, `skip_risk`) |
| `ReportFilter`, `matches_filter`, `nodeid_class` | Marker/class filtering helpers |

### Explanations and citations

| Symbol | Purpose |
|--------|---------|
| `explain_recommendation(changed_paths, store, …)` | Tuple of `TestDecisionExplanation` |
| `explain_with_optional_llm(…, client)` | Explanations plus optional LLM text |
| `build_test_explanation(store, paths, target, decision)` | One explanation |
| `gather_citations(store, paths, target)` | `HistoryCitation` rows from SQLite |
| `format_explanations_text(explanations)` | Plain-text rendering |
| `enhance_explanations_with_llm(…, client)` | LLM narrative from citations only |

### LLM (optional, caller credentials)

| Symbol | Purpose |
|--------|---------|
| `llm_config_from_env()` | `LLMConfig` or `None` |
| `client_from_env()` | `OpenAICompatibleClient` or `None` |
| `OpenAICompatibleClient`, `LLMConfig` | OpenAI-compatible HTTP client (stdlib) |
| `LLMCompletionError` | Request / response errors |

### CI stats

| Symbol | Purpose |
|--------|---------|
| `build_aggregated_ci_stats(store)` | `AggregatedCiStats` |
| `export_ci_stats_json(store)` | JSON string |
| `write_ci_stats_file(path, store)` | Write JSON file |
| `upload_ci_stats(store, endpoint_url=…, token=…)` | HTTPS POST |
| `upload_ci_stats_from_env(store)` | Upload if env configured; else `False` |
| `hash_nodeid(nodeid)` | De-identified hash (not for display of raw ids in export) |
| `CiUploadError` | Upload failures |

### CLI entry point

| Command | Purpose |
|---------|---------|
| `testtime run` | Print recommended pytest command |
| `testtime report` | Interactive or `--no-interactive` report |
| `testtime explain` | Citations; optional `--llm` |
| `testtime ci-export` | De-identified JSON; optional `--upload` |
| `testtime health` | Deployment health JSON (`status`, `version`, `distribution`) |

Exit codes: **0** success, **1** failure, **2** usage, **3** missing optional config.

## Configuration

Set variables in the environment (names only in `.env.example`; do not commit values).

| Variable | Required | Purpose |
|----------|----------|---------|
| `PYPI_PACKAGE_NAME` | No (recommended in CI) | Overrides distribution name; defaults to installed package metadata |
| `TESTPULSE_STORE` | No | SQLite path (default `.testpulse/store.sqlite3`) |

### Third-party keys (optional; you provide your own)

| Variable | Feature | Where to get it |
|----------|---------|-----------------|
| `TESTPULSE_LLM_API_KEY` or `OPENAI_API_KEY` | LLM explanations (`testtime explain --llm`) | Your OpenAI or compatible provider account |
| `TESTPULSE_LLM_BASE_URL` | LLM API base URL | Provider docs (default OpenAI v1 URL) |
| `TESTPULSE_LLM_MODEL` | Chat model id | Provider model list (e.g. `gpt-4o-mini`) |
| `TESTPULSE_STATS_UPLOAD_URL` | CI stats HTTPS upload | Your webhook or ingest endpoint |
| `TESTPULSE_STATS_UPLOAD_TOKEN` | Bearer token for upload | Same service that issued the URL |
| `TESTPULSE_STATS_SALT` | Salt for hashed ids in CI export | Any secret string you choose (default internal salt if unset) |

GitHub Actions can store upload URL/token as repository secrets for `.github/workflows/testpulse-stats.yml`.

## Deployment contract

After build, run `bash scripts/bootstrap_deployment_env.sh` (or use the manifest `install` command) so
`PYPI_PACKAGE_NAME` is set when infrastructure did not inject it. Smoke checks:
`python scripts/verify_deployment_environment.py` and `testtime health`.


TestPulse ships as a **PyPI package** with a CLI-first workflow and an optional stdlib HTTP API for deployment checks:

- `testtime health` — CLI health JSON  
- `testtime serve` — `GET /api/health`, `POST /api/auth/sign-up`, `POST /api/auth/sign-in`, protected `GET /api/me` (set `JWT_SECRET`)  
- `python scripts/deployment_environment_verification.py` — Factory env verification (loads committed `deployment.env`)

| Contract item | TestPulse behavior |
|---------------|-------------------|
| Install / build | `uv sync`, `uv build`; CI runs the same checks in `.github/workflows/ci.yml` |
| Runtime config | `PYPI_PACKAGE_NAME` optional at runtime; falls back to installed wheel / `pyproject.toml` name |
| Health / smoke | `testtime health` (JSON), `testtime --help`, and `uv run pytest` in CI |
| Optional upload | De-identified CI stats only when you set upload URL + token |

## Run it locally

Requires **Python 3.11+**, **git**, and **uv**.

```bash
export PYPI_PACKAGE_NAME=testpulse
python3 scripts/sync_project_name.py
uv sync
```

Record history and get a recommendation:

```bash
uv run pytest --testpulse-record
uv run testtime run
uv run testtime report --no-interactive
```

Development checks:

```bash
uv run ruff check .
uv run mypy src
uv run pytest
uv build
```

Install the console script: `uv sync` exposes **`testtime`** in the project virtualenv.

## Architecture

| Layer | Technology |
|-------|------------|
| Frontend | None (static SVG brand assets only) |
| Backend / HTTP API | None |
| Database | Local SQLite via stdlib (`testpulse.store`) |
| Vector database | None |
| Package manager | uv |
| CLI | `testtime` → `testpulse.cli` |
| Pytest integration | Entry point `testpulse` → `testpulse.recording` |

### Repository layout

```text
src/testpulse/
  api.py              # Public re-exports
  cli.py              # testtime subcommands
  store.py            # SQLite schema and queries
  recording.py        # Pytest hooks
  git_diff.py         # git diff → changed paths
  heuristics.py       # Static path → test mapping
  recommend.py        # Merge heuristics + co-occurrence → pytest command
  flakiness.py        # Bayesian flaky scores
  risk.py             # Skip / regret risk
  regret_hits.py      # Temporal pass-then-fail analysis
  report.py           # Report snapshot and TUI
  citations.py        # SQLite citation records
  explanations.py     # Template + optional LLM explanations
  llm_client.py       # Optional OpenAI-compatible client
  ci_stats.py         # De-identified CI export/upload
  paths.py, config.py # Store path and PYPI_PACKAGE_NAME
tests/                # Unit and CLI tests
scripts/              # sync_project_name.py for PYPI_PACKAGE_NAME → pyproject name
.github/workflows/    # CI and optional TestPulse stats artifact
frontend/public/      # logo.svg, favicon.svg
```

### How components connect

```text
  pytest --testpulse-record
           │
           ▼
  recording.py ──► Store (SQLite) ◄── git_diff.py (changed paths per run)
           │              │
           │              ├── learn change_test_links (co-occurrence)
           │              └── test_results (timing, outcome, marker names)
           │
  testtime run / report / explain / ci-export
           │
           ├── recommend.py ← heuristics + co-occurrence + flakiness
           ├── risk.py, regret_hits.py, report.py, explanations.py
           └── ci_stats.py → JSON artifact (optional HTTPS upload)
```

All learning and recommendations use **your machine’s SQLite file** only; nothing is sent unless you opt into LLM or CI upload with **your** credentials.

## License

MIT
