Metadata-Version: 2.4
Name: steve-qa-tester
Version: 1.0.0
Summary: Steve: Visual QA Agent — Figma vs Built comparison engine
License: MIT
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: google-genai>=0.8.0
Requires-Dist: mcp>=1.0.0
Requires-Dist: pixelmatch>=0.3.0
Requires-Dist: pillow>=10.0.0
Requires-Dist: requests>=2.31.0
Requires-Dist: python-dotenv>=1.0.0
Dynamic: license-file

# Steve — Visual QA Agent

> **Steve** (Systematic Testing & Evaluation Verification Engine) compares your live web build against a Figma design frame and reports exactly what doesn't match — at the pixel, property, and semantic level.

[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](./LICENSE)
[![Python 3.11+](https://img.shields.io/badge/Python-3.11%2B-blue.svg)](https://www.python.org/)

---

## How It Works

```
Figma Node (design source of truth)          Your Live URL
        │                                         │
        ▼ Stage 1                                 ▼ Stage 2
 ┌─────────────────┐                    ┌──────────────────────┐
 │  figma-developer │                    │   Playwright MCP     │
 │  -mcp (PAT auth) │                    │   (browser capture)  │
 │                  │                    │                      │
 │  • Rendered PNG  │                    │  • Full-page load    │
 │  • Property tree │                    │  • Selector wait     │
 │    (colors,fonts,│                    │  • PNG screenshot    │
 │     sizes, etc.) │                    │  • Computed CSS      │
 └────────┬─────────┘                    └──────────┬───────────┘
          │                                         │
          └────────────────┬────────────────────────┘
                           ▼ Stage 3
              ┌────────────────────────────┐
              │      Three-Layer Diff      │
              ├────────────────────────────┤
              │ 1. Pixel Diff             │
              │ 2. Property Diff (CIEDE2000│
              │    color delta, font/size) │
              │ 3. Vision Diff (Gemini)   │
              └────────────┬───────────────┘
                           ▼ Stage 4
                   Merged Findings List
                   (severity + fix hint)
```

- **Figma capture** — uses [`figma-developer-mcp`](https://www.npmjs.com/package/figma-developer-mcp) over stdio with your Personal Access Token. No OAuth, no rate-limit issues.
- **Browser capture** — uses `@playwright/mcp` to navigate, wait for full DOM load, screenshot, and read computed CSS.
- **Pixel diff** — raw pixel-level drift scanning via `pixelmatch`.
- **Property diff** — font size, weight, border radius, padding, and perceptually uniform **CIEDE2000** color delta.
- **Vision diff** — multimodal **Gemini 2.5 Flash** compares both screenshots and lists semantic discrepancies.
- **Reporter** — deduplicates findings, assigns severity (`High / Medium / Low`), optionally files GitHub issues.

---

## Prerequisites

| Requirement | Scope/Role | Where to Get |
|---|---|---|
| **Python 3.11+** | Main engine runtime | [python.org](https://www.python.org/downloads/) |
| **Node.js 18+** | Runs Figma & Playwright MCP drivers | [nodejs.org](https://nodejs.org/) |
| **Figma Access Token** | Reads design specs & captures assets | [Figma Settings](#1-getting-your-figma-personal-access-token) |
| **Gemini API Key** | Compares mock vs actual using Vision AI | [Google AI Studio](#2-getting-your-gemini-api-key) |
| **GitHub Token** *(optional)* | Automatically creates visual QA issue reports | [GitHub Developer Settings](#3-getting-your-github-personal-access-token-optional) |

---

## How to Get Your API Credentials

Follow these steps to obtain the necessary credentials before running Steve locally:

### 1. Getting your Figma Personal Access Token
Steve uses a Personal Access Token (PAT) to connect to your design files:
1. Open **Figma** in your browser or desktop app.
2. Click your profile icon/avatar in the top-left corner and open **Settings**.
3. Select the **Security** tab.
4. Scroll down to the **Personal Access Tokens** section.
5. Click **Generate new token**, enter a name (e.g., `Steve Visual QA`), and check the following scope:
   * **`File content` (Read)** - *required to read nodes & download images*.
6. Copy the generated token immediately and save it in your `.env` file under `FIGMA_ACCESS_TOKEN`.

### 2. Getting your Gemini API Key
Steve uses Gemini 2.5 Flash to perform intelligent, semantic layout comparison:
1. Navigate to [Google AI Studio](https://aistudio.google.com/).
2. Log in with your Google Account.
3. Click **Get API Key** in the top-left sidebar.
4. Click **Create API Key**, select or create a project, and copy the new key.
5. Save this key in your `.env` file under `GEMINI_API_KEY`.

### 3. Getting your GitHub Personal Access Token (Optional)
If you want Steve to automatically file bug report tickets for visual differences:
1. Go to your GitHub account and open **Settings** → **Developer Settings**.
2. Select **Personal Access Tokens** → **Tokens (classic)**.
3. Click **Generate new token (classic)**, name it, and check:
   * **`repo`** (Full control of private & public repositories).
4. Save this token in your `.env` file under `GITHUB_TOKEN` and specify your target repository under `STEVE_GITHUB_REPO=owner/name`.


---

## Setup

**1. Clone and enter the project:**
```bash
git clone https://github.com/your-org/Steve-QA-Tester.git
cd Steve-QA-Tester
```

**2. Create your `.env` file:**
```bash
cp .env.example .env
```

Fill in `.env`:
```env
FIGMA_ACCESS_TOKEN=your_figma_personal_access_token
GEMINI_API_KEY=your_gemini_api_key

# Optional — for filing GitHub tickets
GITHUB_TOKEN=your_github_token
STEVE_GITHUB_REPO=owner/repo
```

**3. Install Python dependencies:**
```bash
pip install -r requirements.txt
pip install -e .
```

> The `pip install -e .` registers the `steve` CLI command globally in your Python environment.

**4. Install Playwright browsers** (one-time):
```bash
npx playwright install chromium
```

> `figma-developer-mcp` and `@playwright/mcp` are installed automatically via `npx -y` on first run — no manual npm install needed.

---

## Run Command

```bash
steve run --url <YOUR_BUILD_URL> --figma <FILE_KEY>:<NODE_ID> --selector "<CSS_SELECTOR>"
```

**Example:**
```bash
steve run \
  --url https://staging.yourapp.com/checkout \
  --figma i6V4j6XDmc6QL6796C4uMu:308-939 \
  --selector "#checkout-btn"
```

You can also paste a full Figma URL:
```bash
steve run \
  --url https://staging.yourapp.com \
  --figma "https://www.figma.com/design/i6V4j6XDmc6QL6796C4uMu/App?node-id=308-939" \
  --selector "body"
```

---

## All CLI Options

```
steve run [OPTIONS]

  --url        URL      Target build URL to screenshot (required)
  --figma      REF      Figma reference: 'file_key:node_id' or full Figma URL (required)
  --selector   STR      CSS selector of the element to compare (default: body)
  --source     URL      GitHub issue URL as source context (optional)
  --dry-run             Skip ticket filing, save reports locally only
  --no-cache            Bypass local Figma cache and force a fresh API fetch

steve replay <RUN_ID>   Replay and print the summary of a past run

steve --help            Show all options
```

---

## Output

On each run Steve prints:

```
┌─ 🎨 Visual Agent Status ───────────────────┐
│ ✓ Completed Visual check. Findings: 3      │
└─────────────────────────────────────────────┘

🔍 Visual Findings:
  1. [🔴 HIGH] #checkout-btn — background_color_mismatch
       Expected : rgb(0, 122, 255)
       Actual   : rgb(255, 0, 0)
       Fix      : Set background-color to #007AFF

  2. [🟡 MEDIUM] #checkout-btn — font_size_mismatch
       Expected : 16px
       Actual   : 14px
       Fix      : Set font-size to 16px

┌─ Summary ───────────────────────────────────┐
│ 🔴 HIGH    1   visual issues                │
│ 🟡 MEDIUM  2   visual issues                │
│ 🟢 LOW     0   visual issues                │
└─────────────────────────────────────────────┘
```

Run artifacts are saved to `runs/<run_id>/`:
- `visual_findings.json` — structured findings
- `expected.png` — Figma design screenshot
- `actual.png` — live page screenshot
- `diff.png` — pixel diff overlay
- `orchestrator_summary.json` — full run summary

---

## Project Structure

```
steve/
├── visual_agent/       Core comparison pipeline
│   └── agent.py
├── reporter_agent/     Deduplicates findings, files tickets
│   └── agent.py
├── orchestrator/       Routes and drives the visual run
│   └── agent.py
├── mcp_client.py       Figma MCP + Playwright MCP clients
├── config.py           Environment configuration
├── cli.py              CLI entry point
└── llm_client.py       Groq LLM client (optional enrichment)
```

---

## Security

- **Read-only** — Steve never writes to your Figma files or codebase.
- **Prompt-injection safe** — Vision model instructions treat screenshots as pure visual data only.
- **No hardcoded secrets** — all credentials are loaded from `.env`, never committed.

---

## License

[MIT](./LICENSE) © 2026 Steve QA Tester
