Metadata-Version: 2.4
Name: test-reporting
Version: 4.4.2
Summary: Multi-project test reporting dashboard — collect results locally or push to S3
Home-page: https://github.com/amahdy77/test-reporting.git
Author: Ashfaqur Mahdy
Author-email: amahdy00@gmail.com
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Topic :: Software Development :: Testing
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: License :: OSI Approved :: MIT License
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pytest>=8.0.0
Provides-Extra: s3
Requires-Dist: boto3>=1.26.0; extra == "s3"
Provides-Extra: playwright
Requires-Dist: playwright>=1.40.0; extra == "playwright"
Provides-Extra: reports
Requires-Dist: python-pptx>=0.6.21; extra == "reports"
Requires-Dist: openpyxl>=3.1.0; extra == "reports"
Requires-Dist: reportlab>=4.0.0; extra == "reports"
Requires-Dist: Pillow>=10.0.0; extra == "reports"
Dynamic: author
Dynamic: author-email
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: license-file
Dynamic: provides-extra
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

# test-reporting

Multi-project test reporting dashboard. Collects pytest results, publishes to a central store (local folder or S3), and serves a clean dashboard anyone can read.

---

## How it works

```
Project A: pytest → local SQLite → publish → ┐
Project B: pytest → local SQLite → publish → ┼─► Central store (local/S3) ─► Dashboard
Project C: pytest → local SQLite → publish → ┘
```

**Local (per project):**
- Tests run and results are saved to a local SQLite database (`test_results.db`)
- SQLite is a scratchpad for local commands (`stats`, `suites`, `cleanup`)
- SQLite is **never** pushed to the store

**Central store (shared):**
- `test-report publish` reads from SQLite and publishes JSON files to the store
- Store contains: run files, artifacts (screenshots/traces), and `data.json`
- Dashboard reads from the store and aggregates all projects

The HTML files are static — deployed once. The `data.json` is rebuilt after every publish.

---

## Installation

```bash
pip install test-reporting

# With S3 support
pip install test-reporting[s3]
```

---

## Quick start

### 1. Add `reporting.ini` to your project

```ini
[reporting]
project_name = my-project

store_type = local
store_path = /shared/test-reports

# auto_publish = true   # push results automatically after every test run
```

### 2. Deploy the dashboard (once, from any project)

```bash
test-report deploy
```

This copies the static HTML files to `store_path`. Only needs to be done once.

### 3. Run tests and publish

```bash
pytest
test-report publish
```

Or set `auto_publish = true` in `reporting.ini` to skip the manual publish step.

### 4. Open the dashboard

```bash
# Local
open /shared/test-reports/index.html

# S3 — your bucket's static website URL
```

---

## Multi-project setup

Every project has its own `reporting.ini` with a unique `project_name`, all pointing to the **same** `store_path` or S3 bucket.

```
Project A:  project_name = frontend-tests,  store_path = /shared/reports
Project B:  project_name = api-tests,       store_path = /shared/reports
Project C:  project_name = e2e-tests,       store_path = /shared/reports
```

The dashboard aggregates all three automatically.

---

## Test suite structure

The plugin detects the suite name from the pytest argument automatically:

```bash
pytest tests/abc/test_suite_abc.py   # suite_name = test_suite_abc
pytest tests/abc/test_file_a.py      # suite_name = test_file_a
```

Suites run on the same day are grouped as a regression in the dashboard.

### Tagging regressions explicitly

Set `REGRESSION_TAG` as an environment variable when kicking off your suites. All runs with the same tag are grouped together:

```bash
# Jenkins — set in each job's environment
REGRESSION_TAG=sprint-42 pytest tests/abc/test_suite_abc.py

# Local
export REGRESSION_TAG=sprint-42
pytest tests/abc/test_suite_abc.py
pytest tests/def/test_suite_def.py
```

Or set it in `reporting.ini`:
```ini
regression_tag = sprint-42
```

---

## Jenkins setup

### Local store (shared network drive)

```ini
[reporting]
project_name = my-project
store_type = local
store_path = //network-share/test-reports
```

```groovy
stage('Test') {
    sh 'pytest tests/'
}
stage('Publish') {
    sh 'test-report publish'
}
```

### S3

```ini
[reporting]
project_name = my-project
store_type = s3
s3_bucket = my-test-reports-bucket
s3_prefix = reports/
s3_region = us-east-1
```

AWS credentials are read from environment variables or an IAM role — not stored in `reporting.ini`.

```groovy
environment {
    REGRESSION_TAG = "2026-05-29"
    AWS_ACCESS_KEY_ID     = credentials('aws-access-key')
    AWS_SECRET_ACCESS_KEY = credentials('aws-secret-key')
}

stage('Test') {
    sh 'pytest tests/'
}
stage('Publish') {
    sh 'test-report publish'
}
```

---

## Configuration reference

```ini
[reporting]
# Required
project_name = my-project

# Store
store_type = local          # local | s3
store_path = /shared/reports  # for store_type = local
# s3_bucket = my-bucket     # for store_type = s3
# s3_prefix = reports/
# s3_region = us-east-1

# Auto-publish after every test run (default: false)
auto_publish = false

# Regression grouping tag (can also be set via REGRESSION_TAG env var)
# regression_tag = sprint-42

# Local SQLite — used by stats/suites/cleanup commands
db_path = test_results.db
retention_days = 365

# Dashboard data retention (affects data.json size and loading speed)
dashboard_max_days = 90           # Days of history in dashboard (default: 90)
dashboard_max_runs_per_suite = 50 # Run history per suite (default: 50)
```

**Note:** `retention_days` controls manual cleanup commands, while `dashboard_max_days` and `dashboard_max_runs_per_suite` control what appears in the dashboard automatically. Individual run files remain accessible regardless of these settings.

---

## CLI commands

| Command | Description |
|---|---|
| `test-report publish` | Push latest test run to the store |
| `test-report deploy` | Deploy static dashboard HTML to the store (run once) |
| `test-report open` | Open local dashboard in browser |
| `test-report stats` | Show stats from latest run (reads local SQLite) |
| `test-report suites` | Show suite statistics (reads local SQLite) |
| `test-report cleanup` | Delete old runs from local SQLite |
| `test-report cleanup-artifacts` | Delete old screenshots and traces (preserves data) |
| `test-report monthly-report` | Generate a project report (PDF default; PPTX/XLSX optional) |
| `test-report monthly-ui` | Open local project and date picker for reports |

### Cleaning up old data

Both cleanup commands use the `retention_days` setting from `reporting.ini` (default: 365 days).

#### `test-report cleanup`
Removes **all test data** from the **local SQLite database only**:
- Deletes test runs, results, steps, logs, and analytics older than `retention_days`
- Use this to free up local database space
- **Does not affect the published store** (JSON files, artifacts, or dashboard data remain intact)
- Only impacts local commands like `stats` and `suites`

#### `test-report cleanup-artifacts`
Removes **only screenshots and traces** while preserving all test data:

```bash
test-report cleanup-artifacts
```

This will:
- Remove screenshots (`.png`) and traces (`.zip`) older than `retention_days`
- Clean **local** directories (`screenshots/`, `traces/`)
- Clean **published store** artifacts (`store_path/artifacts/` for local stores)
- Show file count and total size before deletion
- Ask for confirmation
- **Preserve all local SQLite data and published JSON files** (logs, error messages, analytics remain intact)

**Why use this?** Traces can be 10-50 MB each. This frees disk space on both local machines and the published store while keeping all test data for trend analysis.

**What happens in the UI?** The dashboard gracefully handles missing artifacts:
- Screenshots show: "Screenshot cleaned up" (dashed border placeholder)
- Trace links show: "Trace file cleaned up" (yellow badge)
- All test data, logs, and error messages remain visible

**Note:** Only failed tests have screenshots and traces saved. Passed tests don't generate these artifacts.

**S3 limitation:** Currently only cleans local artifacts. S3 artifact cleanup will be added in a future version.

Configure retention in `reporting.ini`:
```ini
retention_days = 90  # Keep 90 days of artifacts and data
```

---

## Dashboard pages

**`index.html`** — All projects at a glance. Pass rate, health status, last run time.

**`project.html?p=<name>`** — Single project view. Regression history grouped by tag or date, suite run history, trend chart.

**`run.html?r=<id>&p=<name>`** — Run detail. Test file breakdown, every failing test with its error message, screenshot and trace links.


---

## Monthly Reports

Generate a report for one project from its published S3 or local run history. PDF is the default stakeholder report; PowerPoint and Excel are optional exports. The local UI offers rolling date presets, all published history, and custom dates. Reports are downloaded locally and do not change the publishing pipeline.

For AWS SSO setup, commands, date selection, and metric definitions, see [MONTHLY_REPORTS.md](MONTHLY_REPORTS.md).

---

## Migrating from v1

1. `pip install --upgrade test-reporting`
2. Add `store_type`, `store_path` (or S3 config) to `reporting.ini`
3. Run `test-report deploy` once
4. Replace `test-report generate` with `test-report publish` in your pipeline

Old commands that still work: `stats`, `suites`, `cleanup`

---

## Project structure

```
reporting/
  plugin.py       — pytest plugin (auto-collects results)
  publisher.py    — writes run files + rebuilds data.json
  config.py       — reads reporting.ini / env vars
  classifier.py   — categorises failures
  storage.py      — local SQLite
  cli.py          — CLI entry point
  templates/
    index.html    — projects landing page
    project.html  — per-project view
    run.html      — run detail view
```

---

## Requirements

- Python >= 3.9
- pytest >= 8.0.0
- boto3 >= 1.26.0 (only for S3 — `pip install test-reporting[s3]`)

---

## License

MIT
