Metadata-Version: 2.4
Name: fusiontest
Version: 0.2.45
Summary: AI-powered UI testing for mobile and desktop — by FusionLeap.io
Author-email: FusionLeap <hello@fusionleap.io>
License: MIT
Project-URL: Homepage, https://fusionleap.io
Project-URL: Repository, https://github.com/mohigo/fusion-test
Project-URL: Issues, https://github.com/mohigo/fusion-test/issues
Keywords: testing,AI,mobile,desktop,LLM,automation,QA
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Topic :: Software Development :: Testing
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: click>=8.1
Requires-Dist: pyyaml>=6.0
Requires-Dist: httpx>=0.27
Requires-Dist: python-dotenv>=1.0
Requires-Dist: rich>=13.0
Requires-Dist: pydantic>=2.0
Requires-Dist: openai>=1.58
Provides-Extra: mobile
Requires-Dist: Appium-Python-Client>=3.1; extra == "mobile"
Requires-Dist: selenium>=4.18; extra == "mobile"
Provides-Extra: desktop
Requires-Dist: playwright>=1.43; extra == "desktop"
Provides-Extra: mcp
Requires-Dist: mcp<3,>=2.3; extra == "mcp"
Provides-Extra: windows
Requires-Dist: Appium-Python-Client>=3.1; extra == "windows"
Requires-Dist: selenium>=4.18; extra == "windows"
Provides-Extra: macos
Requires-Dist: pyobjc-framework-ApplicationServices>=10; sys_platform == "darwin" and extra == "macos"
Requires-Dist: pyobjc-framework-Cocoa>=10; sys_platform == "darwin" and extra == "macos"
Requires-Dist: pyobjc-framework-Quartz>=10; sys_platform == "darwin" and extra == "macos"
Provides-Extra: mpnet
Requires-Dist: sentence-transformers>=2.7; extra == "mpnet"
Requires-Dist: torch>=2.2; extra == "mpnet"
Provides-Extra: api
Requires-Dist: fastapi>=0.111; extra == "api"
Requires-Dist: uvicorn[standard]>=0.29; extra == "api"
Requires-Dist: pydantic[email]>=2.0; extra == "api"
Requires-Dist: python-jose[cryptography]>=3.3; extra == "api"
Requires-Dist: cryptography>=42; extra == "api"
Requires-Dist: passlib[bcrypt]>=1.7; extra == "api"
Requires-Dist: psycopg2-binary>=2.9; extra == "api"
Requires-Dist: boto3>=1.34; extra == "api"
Provides-Extra: training
Requires-Dist: sentence-transformers>=2.7; extra == "training"
Requires-Dist: torch>=2.2; extra == "training"
Requires-Dist: datasets>=2.19; extra == "training"
Requires-Dist: accelerate>=0.30; extra == "training"
Requires-Dist: scikit-learn>=1.4; extra == "training"
Requires-Dist: pandas>=2.2; extra == "training"
Requires-Dist: numpy>=1.26; extra == "training"
Requires-Dist: tqdm>=4.66; extra == "training"
Requires-Dist: wandb>=0.17; extra == "training"
Provides-Extra: openai
Requires-Dist: openai>=1.58; extra == "openai"
Provides-Extra: groq
Requires-Dist: openai>=1.58; extra == "groq"
Provides-Extra: claude
Requires-Dist: anthropic>=0.25; extra == "claude"
Provides-Extra: gemini
Requires-Dist: google-genai>=1.0; extra == "gemini"
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: pytest-asyncio>=0.23; extra == "dev"
Requires-Dist: pytest-cov>=5.0; extra == "dev"
Requires-Dist: ruff>=0.4; extra == "dev"
Requires-Dist: mypy>=1.10; extra == "dev"
Requires-Dist: pre-commit>=3.7; extra == "dev"
Requires-Dist: fusiontest[desktop,mobile,openai]; extra == "dev"
Provides-Extra: all
Requires-Dist: fusiontest[api,claude,desktop,gemini,groq,macos,mobile,mpnet,openai,training,windows]; extra == "all"

# 🧪 FusionTest

> AI-powered UI testing for web, desktop and mobile — built by [FusionLeap.io](https://fusionleap.io)

FusionTest runs UI tests written as plain-English goals. An LLM reads the
screen and decides what to do, an independent judge model confirms each goal
from the screen, a screenshot and the app's own errors, and goals that pass are
recorded so later runs replay them without calling the AI at all.

Coding agents use it to verify their changes: `fusiontest verify` (or the MCP
server) reads the diff, checks what it promises in the running app, and keeps
what passes as regression goals. See [docs/coding-agents.md](docs/coding-agents.md).

---

## ✨ Features

| Feature | Notes |
|---|---|
| Natural-language goals | `"Open the Pricing page and verify all three plans are shown"` |
| Verify a change | `fusiontest verify` plans checks from a diff and judges them; JSON verdict for agents and CI |
| MCP server | `fusiontest mcp`: `verify_change`, `run_goals`, `scan_site` for Claude Code and other agents |
| Independent judge | A stronger model judges pass/fail (Claude or OpenAI when set) while a cheap one clicks |
| Goal discovery | Crawls a site and proposes goals grounded in its real content |
| Record & replay | Passing goals replay with no LLM calls; the AI repairs them when the page changes |
| Honest results | Capacity or infrastructure problems are reported as **errors**, not test failures |
| Multi-provider LLMs | Groq, Claude, Gemini, OpenAI or local Ollama, with automatic fallback |
| Safe secrets | `{{PASSWORD}}` placeholders — the AI never sees values, which are only typed on allowed sites |
| Guardrails | Invalid-action checks, loop detection, backtracking, no-progress detection |
| Reporting | JSON reports, dashboard uploads, failure screenshots, Slack, GitHub Actions annotations, webhooks |
| Platforms | Web (Playwright), Electron, Android & iOS (Appium), Windows, macOS |

---

## 🏗 Architecture

```
fusiontest/
├── core/         # Runner, LLM action model + fallback chain, goal verifier,
│                 # screen parser, secrets, replay, SSRF guard (netguard)
├── desktop/      # Playwright (web/Electron), Windows, macOS adapters
├── mobile/       # Android (UIAutomator2) and iOS (XCUITest) adapters via Appium
├── discovery/    # Goal discovery from a site's pages
├── guardrails/   # Action validation, loop detection, backtracking
├── reporting/    # JSON reports, Slack, GitHub annotations
└── training/     # Experimental MPNet fine-tuning (not wired into runs)
api/              # SaaS backend (FastAPI + PostgreSQL) — see docs/api.md
dashboard/        # React dashboard
website/          # Marketing site (Next.js)
```

Design decisions are recorded in [docs/adr](docs/adr) — start with
[ADR-007](docs/adr/ADR-007-llm-capacity-and-execution.md).

---

## 🚀 Quickstart

```bash
# macOS (Homebrew Python) or anywhere you want an isolated CLI — recommended
brew install pipx && pipx ensurepath        # then open a new terminal
pipx install "fusiontest[desktop]"
"$(pipx environment --value PIPX_LOCAL_VENVS)/fusiontest/bin/playwright" install chromium

# …or inside a virtual environment
python3 -m venv .venv && source .venv/bin/activate
pip install "fusiontest[desktop]"
playwright install chromium

export GROQ_API_KEY=gsk_...          # or ANTHROPIC_API_KEY / GEMINI_API_KEY / OPENAI_API_KEY

fusiontest run --url https://example.com \
  --goal "Verify the page heading says Example Domain" --headless

# Changed the UI? Check that the change does what it says (uncommitted changes by default)
fusiontest verify --url http://localhost:5173

# No goals yet? Map what works on a test/staging site, and save the flows that do
fusiontest scan https://staging.example.com --save fusiontest.goals.yaml
# Behind a login: EMAIL and PASSWORD come from the environment, never the AI
fusiontest scan https://staging.example.com --login "Sign in with email {{EMAIL}} and password {{PASSWORD}}"
```

Groq, OpenAI and Ollama work out of the box; for Claude or Gemini add the
extra — `fusiontest[desktop,claude]` or `fusiontest[desktop,gemini]`.
Upgrade with `pipx upgrade fusiontest` (or `pip install -U fusiontest` in
the venv).

Set `FUSIONTEST_API_KEY` (from the dashboard's API Keys page) to upload each
run to your FusionTest dashboard.

Exit codes: `0` all goals passed · `1` a goal failed · `2` some goals couldn't
be evaluated (LLM capacity or infrastructure) — not a test failure.

> **Full guide:** [docs/getting-started.md](docs/getting-started.md) — platforms,
> goal files, configuration, secrets, replay, dashboard and CI.

---

## 📝 Writing goals

```yaml
# tests/goals/smoke.yaml
name: Marketing site smoke test
platform: playwright
url: https://example.com
goals:
  - "Open the Pricing page and verify three plan tiers are shown"
  - "Open the Contact page, submit the form empty, and verify required-field errors appear"
  - name: Log in
    goal: "Log in with {{EMAIL}} and {{PASSWORD}} and verify the dashboard loads"
```

```bash
fusiontest run --file tests/goals/smoke.yaml --headless
```

Tips: say what to verify, not just where to go; one outcome per goal; don't
submit real forms on production sites.

---

## 🛠 Configuration

`fusiontest init` creates `fusiontest.config.yaml`:

```yaml
backend: groq               # unset = auto-detect from API keys
guardrails:
  loop_detection_window: 5
  max_retries: 3
replay:
  enabled: true
  trace_dir: .fusiontest/traces
  reverify_every: 10
```

---

## 🔌 CI/CD

Run `fusiontest run --headless` in any CI system. With `FUSIONTEST_API_KEY`
set, replay recordings are shared through your FusionTest account (laptop, CI
and dashboard all replay each other's passing goals); without it, cache
`.fusiontest/traces`. Guides for GitHub Actions, Bitrise,
Fastlane and Jenkins: [docs/ci-integrations.md](docs/ci-integrations.md).

---

## 🤝 Contributing

See [CONTRIBUTING.md](CONTRIBUTING.md) and [ROADMAP.md](ROADMAP.md).

## 📄 License

MIT © [FusionLeap.io](https://fusionleap.io)
