Metadata-Version: 2.4
Name: playwright-tc-failure-ai-analyzer
Version: 1.0.0
Summary: Provider-agnostic AI failure-analysis engine and QA assistant that plugs into any pytest + Playwright project. Turns a failing test into an evidence-grounded root-cause analysis and a tracker-ready bug report. Works fully offline with zero API keys.
Author-email: Aman Deep <amandeep242712@gmail.com>
License-Expression: MIT
Project-URL: Homepage, https://github.com/amandeepsdet/playwright-tc-failure-ai-analyzer
Project-URL: Issues, https://github.com/amandeepsdet/playwright-tc-failure-ai-analyzer/issues
Project-URL: Source, https://github.com/amandeepsdet/playwright-tc-failure-ai-analyzer
Keywords: pytest,playwright,qa,test-automation,ai,root-cause-analysis,failure-analysis,bug-report,llm,rag
Classifier: Framework :: Pytest
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Testing
Classifier: Topic :: Software Development :: Quality Assurance
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pytest>=7.0
Provides-Extra: playwright
Requires-Dist: playwright>=1.40; extra == "playwright"
Requires-Dist: pytest-playwright>=0.5; extra == "playwright"
Provides-Extra: reports
Requires-Dist: allure-pytest>=2.13; extra == "reports"
Requires-Dist: pytest-html>=4.1; extra == "reports"
Provides-Extra: openai
Requires-Dist: openai>=1.0; extra == "openai"
Provides-Extra: llm
Requires-Dist: openai>=1.0; extra == "llm"
Requires-Dist: requests>=2.31; extra == "llm"
Provides-Extra: vector
Requires-Dist: chromadb>=0.4; extra == "vector"
Provides-Extra: all
Requires-Dist: playwright>=1.40; extra == "all"
Requires-Dist: pytest-playwright>=0.5; extra == "all"
Requires-Dist: allure-pytest>=2.13; extra == "all"
Requires-Dist: pytest-html>=4.1; extra == "all"
Requires-Dist: openai>=1.0; extra == "all"
Requires-Dist: requests>=2.31; extra == "all"
Requires-Dist: python-dotenv>=1.0; extra == "all"
Dynamic: license-file

# playwright-tc-failure-ai-analyzer

**Drop-in AI failure analysis for any pytest + Playwright project.**

`playwright-tc-failure-ai-analyzer` turns a failing test into an evidence-grounded
**root-cause analysis**, a **confidence score**, a **likely owning team**, and a
**tracker-ready bug report** — then attaches it all to your Allure / pytest-html
reports. It works **fully offline with zero API keys** (deterministic heuristic
engine + pure-Python embeddings + local JSON vector store), and transparently
upgrades to a real LLM when you point it at one.

> Install name: `playwright-tc-failure-ai-analyzer` — import name: `qa_ai_engine`

---

## Install

```bash
pip install playwright-tc-failure-ai-analyzer                      # core (offline heuristic engine)
pip install "playwright-tc-failure-ai-analyzer[playwright,reports]"  # + live page evidence + report attachments
pip install "playwright-tc-failure-ai-analyzer[all]"              # everything, including LLM providers
```

## 1-minute integration

The package ships a **pytest plugin** that is auto-discovered — there is nothing
to import. Just enable it:

```bash
# PowerShell
$env:AI_ENABLED = "true"
pytest

# bash
AI_ENABLED=true pytest
```

Any test that fails and uses the pytest-playwright `page` fixture is analysed
automatically. Console output on failure:

```
AI analysis: A locator did not resolve to a visible element within the timeout.
             (Locator, confidence=80%) — owner=UI Automation / QA team
```

That's it. No `conftest.py` changes required. The plugin is a **no-op** unless
`AI_ENABLED=true`, so it never affects normal runs.

## Use the engine directly (any framework)

```python
from qa_ai_engine import AIEngine

engine = AIEngine()
outcome = engine.analyze_failure(
    test_name="checkout::test_pay",
    exception=err,          # the caught exception
    page=page,              # optional Playwright page for live evidence
    assertion_message=str(err),
)
print(outcome.analysis.root_cause, outcome.analysis.confidence, outcome.analysis.owner)
print(engine.bug_gen.to_markdown(outcome.bug_report))
```

## QA Assistant CLI

```bash
qa-ai status                 # provider / config status
qa-ai analyze-last-failure   # full analysis + bug report for the latest failure
qa-ai release-readiness      # 0-100 go/no-go score from failure history
qa-ai search "login timeout" # RAG search over past failures
qa-ai                        # interactive chat mode
```

## Configuration (all via environment variables)

| Variable | Default | Purpose |
|----------|---------|---------|
| `AI_ENABLED` | `false` | Master switch. |
| `AI_PROVIDER` | `heuristic` | `openai` \| `azure` \| `claude` \| `gemini` \| `ollama` \| `heuristic`. |
| `AI_MODEL` | `gpt-4o-mini` | Model name for the chosen provider. |
| `AI_API_KEY` | – | Key for the chosen provider (or the provider's own env var). |
| `AI_VECTOR_BACKEND` | `json` | `json` (offline) or `chroma`. |
| `AI_BASE_DIR` | current dir | Where `failure_history/`, `vector_db/`, `ai_reports/` are written. |
| `QA_AI_DISABLE_PLUGIN` | `false` | Force the pytest plugin off (e.g. when wiring the engine manually). |

Offline mode (the default) requires **no keys and no extra dependencies**.

## What you get on every failure

- Root cause + failure **category** (Locator, Backend, Auth, Network, Data, …)
- **Confidence** score and **severity**
- **Likely owning team** (configurable routing)
- Recommended fix
- **RAG** search over similar past failures
- A structured, tracker-ready **bug report** (title, steps, priority, owner)
- JSON / Markdown / HTML artifacts + Allure & pytest-html attachments

## Provider setup (opt-in)

All providers are **lazily imported** — you only need the SDK for the one you use.
Set `AI_ENABLED=true` and `AI_PROVIDER`, then supply the provider's credentials.

**OpenAI**
```bash
pip install "playwright-tc-failure-ai-analyzer[openai]"
$env:AI_ENABLED="true"; $env:AI_PROVIDER="openai"
$env:AI_MODEL="gpt-4o-mini"; $env:AI_API_KEY="sk-..."
```

**Azure OpenAI**
```bash
$env:AI_ENABLED="true"; $env:AI_PROVIDER="azure"
$env:AI_API_KEY="<azure-key>"; $env:AI_MODEL="<deployment-name>"
$env:AZURE_OPENAI_ENDPOINT="https://<resource>.openai.azure.com"
```

**Ollama (local, no key)**
```bash
$env:AI_ENABLED="true"; $env:AI_PROVIDER="ollama"
$env:AI_MODEL="llama3"   # OLLAMA_HOST defaults to http://localhost:11434
```

If a provider call fails at runtime, the engine automatically falls back to the
deterministic offline analyzer — a run is never blocked by a missing key or a
network error.

## Teach it about your app (optional)

The assistant answers are grounded in a small, generic knowledge base by
default. Point it at your own application without touching code:

| Variable | Purpose |
|----------|---------|
| `QA_AI_APP_NAME` | Friendly name of the app under test. |
| `QA_AI_FRAMEWORK_CONTEXT` | One-paragraph description used for RAG context. |
| `QA_AI_KNOWLEDGE_FILE` | Path to a JSON file with richer knowledge. |

Example `knowledge.json`:
```json
{
  "app_name": "Acme Checkout",
  "context": "E-commerce checkout built on React + a REST orders API.",
  "widgets": { "cart": "Cart summary panel on the right rail." },
  "apis": { "orders": "POST /api/orders creates an order and returns an id." }
}
```
```bash
$env:QA_AI_KNOWLEDGE_FILE="knowledge.json"
```

## Troubleshooting

| Symptom | Cause / fix |
|---------|-------------|
| Plugin does nothing | `AI_ENABLED` is not `true`. The plugin is a no-op otherwise. |
| `import qa_ai_engine` fails | Package not installed in the active interpreter — `pip install playwright-tc-failure-ai-analyzer`. |
| No live page evidence | Install the `playwright` extra and use the `page` fixture. |
| Analysis is always "heuristic" | No provider configured — set `AI_PROVIDER` + credentials (see above). |
| Double analysis | The engine is wired both via the plugin and manually — set `QA_AI_DISABLE_PLUGIN=true` for manual wiring. |
| Artifacts written to the wrong place | Set `AI_BASE_DIR` to control where `failure_history/`, `vector_db/`, `ai_reports/` go. |

## License

MIT
