Metadata-Version: 2.4
Name: trunft
Version: 0.2.1
Summary: TruNFT - unified Non-Functional Testing platform (CLI + MCP server) for Web, Mobile Native, Mobile Browser and Mobile Hybrid apps
Author-email: Shan Konduru <ShanKonduru@gmail.com>
License-Expression: MIT
Project-URL: Homepage, https://github.com/ShanKonduru/TruNFT
Project-URL: Source, https://github.com/ShanKonduru/TruNFT
Project-URL: Issues, https://github.com/ShanKonduru/TruNFT/issues
Keywords: non-functional-testing,performance-testing,load-testing,security-testing,chaos-engineering,accessibility,compatibility-testing,mobile-testing,k6,mcp
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Testing
Classifier: Topic :: Software Development :: Quality Assurance
Classifier: Topic :: System :: Benchmark
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: PyYAML>=6.0
Provides-Extra: web
Requires-Dist: playwright>=1.45; extra == "web"
Provides-Extra: mobile
Requires-Dist: Appium-Python-Client>=4.0; extra == "mobile"
Provides-Extra: chaos
Requires-Dist: chaostoolkit>=1.19; extra == "chaos"
Provides-Extra: mcp
Requires-Dist: mcp<3,>=2.3; extra == "mcp"
Provides-Extra: all
Requires-Dist: trunft[chaos,mcp,mobile,web]; extra == "all"
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: anyio>=4; extra == "dev"
Requires-Dist: mcp<3,>=2.3; extra == "dev"
Dynamic: license-file

# TruNFT — Unified Non-Functional Testing Platform

One framework, one CLI, one config and one report for **non-functional testing** across:

| Platform | Key | What is exercised |
|---|---|---|
| Web UI | `web` | Site URLs in desktop browsers |
| Mobile browser | `mobile-browser` | Site URLs with mobile form factor / device emulation |
| Mobile native | `mobile-native` | App binary (APK/IPA) + the backend APIs it calls |
| Mobile hybrid | `mobile-hybrid` | App binary + embedded web content + backend APIs |

TruNFT is an **orchestrator**. It sends each kind of test to an established tool (k6, OWASP ZAP, Toxiproxy, Chaos Toolkit, Lighthouse, Playwright, adb/xcrun), applies the same SLO thresholds to the results, and writes JSON, JUnit and HTML reports.

## Coverage

| Pillar (`--pillar`) | Test types (`-t`) | Engine |
|---|---|---|
| **Performance** `performance` | load, stress, spike, endurance (soak), scalability, capacity | k6 (one parameterised script, profiles in config) |
| **Security** `security` | vulnerability, penetration, access_control | OWASP ZAP baseline/full (Docker); built-in role × endpoint authZ matrix |
| **Reliability & availability** `reliability` | stability, failover, recovery, disaster_recovery | Background availability probe + configurable trigger/restore hooks (RTO, outage) |
| **Resilience & chaos** `chaos` | network_latency, service_failure, resource_exhaustion, experiment | Toxiproxy, shell hooks (kubectl / docker / AWS FIS), Chaos Toolkit |
| **Usability & accessibility** `accessibility` | accessibility, usability | Lighthouse (axe-core rules, Core Web Vitals); mobile vs desktop form factor |
| **Compatibility** `compatibility` | browser, device, os, configuration | Playwright (Chromium/Firefox/WebKit, device emulation, locale/dark mode/…) |
| **Installability & portability** `installability` | install, upgrade, uninstall, portability | adb (Android), xcrun simctl (iOS); multi-host health checks |
| **Compliance & localization** `compliance` | security_headers, privacy_cookies, localization | Built-in HTTP checks (OWASP headers, cookie flags, per-locale `lang` and untranslated strings) |

Run `trunft list` for the full platform × test-type matrix.

## Two interfaces, one engine

```
           trunft (CLI)          trunft-mcp (MCP server: stdio / streamable-http)
                 \                      /
                  +---- trunft.api ----+      capabilities, doctor, plan, run, reports
                            |
              runner -> pillars -> external tools (k6, ZAP, Lighthouse, ...)
                            |
               reports/<run>/results.json, junit.xml, report.html
```

Both interfaces read the same config and write the same reports, so a run started from one can be inspected from the other.

## Quick start (CLI)

```bash
python -m venv .venv && .venv/Scripts/activate     # source .venv/bin/activate on Linux/macOS
pip install -e ".[dev]"            # add [web] for Playwright, [mobile] for Appium client
trunft doctor                      # shows which external tools are available
trunft setup                       # installs missing ones (winget/brew/npm/pip); --dry-run to preview
trunft plan -e staging --pillar performance       # preview, runs nothing
trunft run --dry-run               # plan everything, execute nothing
trunft run -e dev -p web --pillar compliance
trunft run -e staging --pillar performance -t load -t spike
trunft reports                     # recent runs
trunft report <run> --failures     # what broke and which thresholds were breached
pytest                             # framework self-tests
```

`list`, `doctor`, `plan`, `reports` and `report` accept `--json`.
Exit code: `0` = no failures, `1` = at least one FAILED/ERROR, `2` = bad config or arguments.

## MCP server (AI agents / IDEs)

```bash
pip install -e ".[mcp]"
trunft-mcp --root C:/myprojects/TruNFT                       # stdio
trunft-mcp --transport streamable-http --port 8765           # http://127.0.0.1:8765/mcp
trunft mcp                                                   # same server via the main CLI
```

Register it with Claude Code:

```bash
claude mcp add trunft --env TRUNFT_ROOT=C:/myprojects/TruNFT -- C:/myprojects/TruNFT/.venv/Scripts/trunft-mcp.exe
```

Or in Claude Desktop / any MCP client config:

```json
{ "mcpServers": { "trunft": {
    "command": "C:/myprojects/TruNFT/.venv/Scripts/trunft-mcp.exe",
    "env": { "TRUNFT_ROOT": "C:/myprojects/TruNFT" } } } }
```

| Tool | Purpose |
|---|---|
| `list_capabilities` | Pillars, test types, required tools, and which platforms each applies to |
| `check_tools` | Which external tools are installed |
| `list_environments` / `get_config` | Environments and the merged config (secrets redacted) |
| `plan_tests` | Preview the platform × pillar × test-type combinations |
| `start_test_run` | Start a background run and return a `run_id` at once. **`dry_run` defaults to `true`** |
| `get_test_run` / `cancel_test_run` | Poll progress and results, or stop the remaining tests |
| `list_test_reports` / `get_test_report` | Past runs from CLI or MCP; `only_failures` filter |

Runs are asynchronous because soak tests can last for hours. The agent starts a run, polls it, then reads the report. Tools that only read are marked read-only. `start_test_run` is marked destructive and open-world because real runs generate load, run security scans and can inject faults.

### Result statuses
| Status | Meaning |
|---|---|
| `passed` | Ran, and every threshold was met |
| `failed` | Ran, but breached a threshold or assertion |
| `error` | Couldn't complete (tool crashed, target unreachable) |
| `skipped` | Dry-run, not applicable to the platform, or a required tool isn't installed |
| `not_implemented` | Placeholder: needs config (e.g. trigger hook) or code; the message says what |

## Layout

```
config/trunft.yaml              master config: targets, thresholds (SLOs), pillar settings
config/environments/*.yaml      per-environment overrides (dev, staging, ...); ${ENV_VAR:-default} supported
src/trunft/api.py               service layer shared by CLI and MCP server
src/trunft/cli.py               `trunft` command line
src/trunft/mcp_server.py        `trunft-mcp` MCP server (stdio / streamable-http)
src/trunft/core/                config, models, pillar base + registry, runner, thresholds, probes
src/trunft/pillars/*.py         one module per NFT pillar (auto-discovered)
src/trunft/reporting/           results.json, junit.xml, report.html
workloads/k6/scenario.js        parameterised k6 workload - put real user journeys here
chaos/experiments/              Chaos Toolkit experiments (*.json)
docker/docker-compose.yml       toxiproxy, influxdb, grafana for local runs
apps/android, apps/ios          app binaries under test (git-ignored)
.github/workflows/ci.yml        unit tests on push/PR (Linux + Windows); no live tests in CI
.github/workflows/publish.yml   gated PyPI release on v* tags
```

## Thresholds (SLO gates)

Every pillar returns metrics. A threshold `max_<metric>` or `min_<metric>` under `thresholds.<pillar>.default`, or under a per-test-type block, decides pass or fail:

```yaml
thresholds:
  performance:
    default: {max_p95_ms: 800, max_error_rate: 0.01}
    stress:  {max_p95_ms: 2000}
```

## Try it against the bundled test app (TruShop)

`sut/` contains a Dockerised system under test that exercises every test type: a load-balanced web tier with a DR replica, a catalog microservice behind Toxiproxy, i18n pages, auth roles and fault-injection hooks. It can also switch on deliberate defects so you can see what each check catches.

```bash
docker compose -f sut/docker-compose.yml up -d --build
trunft run -e sut -p web --pillar compliance --pillar reliability --pillar chaos
```

See [sut/README.md](sut/README.md).

## Installing from PyPI

```bash
pip install trunft            # core; extras: trunft[web], trunft[mcp], trunft[chaos], trunft[all]
trunft init my-nft-project    # creates config/, workloads/k6, chaos/, docker/ from bundled templates
cd my-nft-project && trunft setup && trunft run --dry-run
```

## Releasing

Publishing uses PyPI **Trusted Publishing** (`.github/workflows/publish.yml`, no API tokens):

1. Bump `__version__` in `src/trunft/__init__.py` (the single source of the version) and commit.
2. Tag that commit and push the tag: `git tag v0.2.0 && git push origin v0.2.0`

The workflow uploads only if **all** of these gates pass:

| Gate | Check |
|---|---|
| Version | Tag `v<x.y.z>` == `__version__` == built sdist/wheel versions |
| Unit | `pytest` on Linux + Windows, Python 3.10 and 3.13 |
| Security | Sources compile; Bandit SAST with zero findings; `pip-audit --strict` on the runtime dependency set |
| Build | `twine check --strict`; the wheel is installed into a clean venv, then `trunft init` and a dry run are executed |

Then it publishes to PyPI (GitHub environment `pypi`).
Run the version gate locally with `python scripts/check_release_version.py v0.2.0`.

If you edit files under `config/`, `workloads/`, `chaos/` or `docker/`, copy them to `src/trunft/templates/project/`. A test fails if the two copies drift apart.

## Extending

Add a module under `src/trunft/pillars/`:

```python
from trunft.core.pillar import Pillar, RunContext, register
from trunft.core.models import Status

@register
class Energy(Pillar):
    name, title = "energy", "Battery & Energy"
    test_types = {"drain": "Battery drain over a 30-min journey"}
    type_platforms = {"drain": Platform.mobile_apps()}
    tools = {"drain": ["adb"]}

    def run(self, ctx: RunContext, test_type: str):
        ...
        return self.result(ctx, test_type, Status.PASSED, metrics={"mah_per_hour": 120})
```

## Roadmap: open placeholders (`not_implemented`)

- **Mobile client performance**: startup time, memory, CPU and frame rate (adb `dumpsys`, Xcode Instruments)
- **Mobile security**: MobSF static/dynamic analysis of APK/IPA
- **Native accessibility**: Espresso Accessibility Test Framework, XCUITest `performAccessibilityAudit`
- **Device-cloud compatibility**: Appium or Selenium grid (BrowserStack, Sauce Labs, LambdaTest) for the OS and native matrix
- **Upgrade data retention**, **DR RPO verification**, **client-side network conditioning** (offline/3G)
- **Native localization**: resource-bundle key completeness plus per-locale screenshots
- **Compliance evidence packs**: GDPR, PCI-DSS, HIPAA, WCAG conformance statement

> ⚠️ Run active security scans (`penetration`) and chaos/DR hooks only against environments you are authorised to test. `allow_active_scan` is `false` by default.
