Metadata-Version: 2.4
Name: scriptgini
Version: 1.5.1
Summary: Agentic AI system that converts functional test cases into automation test scripts.
Author: ScriptGini Team
License: Proprietary
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Framework :: FastAPI
Requires-Python: >=3.11
Description-Content-Type: text/markdown

# ScriptGini

> **Enterprise-grade Agentic AI system that converts functional test cases into high-quality, review-ready automation test scripts.**

Current release: v1.5.1 (Sprint 6 coverage-complete patch)

---

## What is ScriptGini?

ScriptGini is an AI-powered test automation engine built for Quality Engineering teams. You feed it a functional test case and an Application Under Test (AUT) URL — it returns a production-ready automation script in your chosen framework, generated by a multi-step LangGraph agent that reasons about test intent before writing a single line of code.

---

## Features

- **Agentic 3-node LangGraph pipeline** — Intent analysis → Script generation → Quality review
- **Multi-provider LLM support** — OpenAI, Ollama (local), OpenRouter, Google Gemini, AWS Bedrock
- **Framework-agnostic output** — Playwright Python, Selenium Python, UFT VBScript, Cypress JS
- **Intelligent selector strategy** — Role → Label → data-testid → CSS → XPath (last resort)
- **Project & AUT management** — Store multiple projects, each with its own base URL and defaults
- **Full test case history** — Every generated script is stored in SQLite with status and token usage
- **Execution history persistence** — Every run is stored in `script_runs` with stdout/stderr, exit code, and duration
- **Durable execution APIs** — Async execution jobs with run/status/abort endpoints and idempotency-key support
- **Hardened execution sandbox** — Script runs use isolated Python mode, static safety validation, and restricted environment variables
- **Bulk job orchestration** — Project-level bulk generate and bulk run with pollable job status
- **Run analytics dashboard** — Project-level pass/fail/timeout metrics and recent failure feed
- **Coverage analytics** — Module-level coverage and execution visibility for test cases
- **Richer test case intake** — Import `.txt`, `.md`, `.json`, `.csv`, `.feature`, `.yml/.yaml`, and `.xlsx`
- **Import preview mapping** — Preview parsed scenarios in the UI before creating a project workspace
- **Script lifecycle tracking** — Refactor, version history, diff, and rollback support
- **REST API** — FastAPI with auto-generated Swagger UI
- **Alembic migrations** — Safe, versioned schema management over SQLite

---

## Tech Stack

| Layer | Technology |
|---|---|
| API | FastAPI + Uvicorn |
| Agentic AI | LangGraph + LangChain |
| LLM Providers | OpenAI, Ollama, OpenRouter, Gemini, Bedrock |
| Database | SQLite |
| ORM | SQLAlchemy 2.0 |
| Migrations | Alembic |
| Config | Pydantic Settings (.env) |

---

## Quick Start

### Windows

```bat
start.bat
```

### Linux / macOS

```bash
chmod +x start.sh
./start.sh
```

The script will:
1. Create a Python virtual environment
2. Install all dependencies
3. Copy `.env.example` → `.env` if missing (edit it before re-running)
4. Run Alembic migrations
5. Start the server and open Swagger UI in your browser

To load a ready-made sample workspace, use the `Load Demo Project` button in the web UI or call `POST /api/v1/demo/load`.

---

## Configuration

Copy `.env.example` to `.env` and fill in the values you need:

```bash
cp .env.example .env
```

```env
# Choose your default provider
DEFAULT_LLM_PROVIDER=openrouter   # openai | ollama | openrouter | gemini | bedrock

# OpenAI
OPENAI_API_KEY=your_openai_api_key_here

# Ollama (local — no key needed)
OLLAMA_BASE_URL=http://localhost:11434
OLLAMA_MODEL=llama3
OLLAMA_NUM_PREDICT=700

# Generation latency controls
LLM_REQUEST_TIMEOUT_SECONDS=45
SCRIPT_GENERATION_TIMEOUT_SECONDS=180
SKIP_REVIEW_FOR_OLLAMA=true

# OpenRouter
OPENROUTER_API_KEY=your_openrouter_api_key_here
OPENROUTER_MODEL=openai/gpt-4o

# Google Gemini
GOOGLE_API_KEY=your_google_api_key_here

# AWS Bedrock
AWS_ACCESS_KEY_ID=your_aws_access_key_id
AWS_SECRET_ACCESS_KEY=your_aws_secret_access_key
AWS_REGION_NAME=us-east-1
```

> `.env` is git-ignored. Never commit real API keys.

If local generation feels slow, reduce `OLLAMA_NUM_PREDICT`, keep `SKIP_REVIEW_FOR_OLLAMA=true`, or switch to a smaller/faster Ollama model.

---

## OpenAPI Specification

**ScriptGini implements a production-ready OpenAPI 3.0.3 specification** that defines a complete enterprise REST API with 50+ endpoints, 60+ schemas, and comprehensive documentation.

### Specification Highlights

**The API specification includes 12 salient features**:

1. **Multi-tenant Architecture** — Organizations, Workspaces, Teams with hierarchical RBAC
2. **Advanced LLM Management** — Multi-provider orchestration, health monitoring, cost tracking, and model governance
3. **Intelligent Test Data Management** — Multi-source ingestion, synthetic generation, PII masking, state locking, placeholder mapping
4. **Asynchronous Job Management** — HTTP 202 Accepted, idempotent execution, job polling, webhooks
5. **Comprehensive Reporting** — Structured logs, artifact storage, signed downloads, execution diagnostics
6. **Analytics & Insights** — Dashboards, trend analysis, flakiness detection, code coverage, LLM usage
7. **Defect Lifecycle** — Auto-detection, Jira/Azure/GitHub sync, severity tracking, traceability
8. **Script Versioning** — Git-style history, quality metrics, refactoring, diff metadata
9. **Webhooks** — Event-driven notifications, delivery guarantees, retry policies
10. **Security & Authorization** — JWT + OAuth2, API keys with scopes, RBAC enforcement, rate limiting
11. **Error Handling & Observability** — Standardized error envelope, request ID tracing, correlation IDs
12. **Pagination & Filtering** — limit/offset/sort, comprehensive filtering, faceted search

### API Reference

Once running, visit:

| URL | Description |
|---|---|
| `http://localhost:8000/docs` | Swagger UI (interactive) |
| `http://localhost:8000/redoc` | ReDoc |
| `http://localhost:8000/health` | Health check |
| `http://localhost:8000/openapi.json` | Raw OpenAPI 3.0.3 specification |

### Full OpenAPI Documentation

- **Complete Specification**: [docs/openapi-improved-draft-2026-05-10.yaml](docs/openapi-improved-draft-2026-05-10.yaml)
- **Feature Documentation**: [docs/OPENAPI-SPECIFICATION.md](docs/OPENAPI-SPECIFICATION.md) — Covers all 12 salient features with examples and flows
- **API Architecture Review**: [docs/api-architecture-review-2026-05-10.md](docs/api-architecture-review-2026-05-10.md) — Analysis of current state vs. target enterprise API

### API Domains (50+ Endpoints)

| Domain | Endpoints | Purpose |
|--------|-----------|---------|
| **Authentication & IAM** | 8 | Login, token refresh, user management, API keys |
| **Organizations & Teams** | 6 | Multi-tenancy, team management, RBAC |
| **Workspaces & Projects** | 5 | Organizational hierarchy, configuration |
| **Test Data Management** | 11 | Data sets, reservations, masking, synthetic generation |
| **LLM Management** | 5 | Provider registration, model governance, cost tracking |
| **Script Engineering** | 7 | Generation, versioning, quality, refactoring |
| **Test Cases** | 4 | CRUD operations for test definitions |
| **Test Orchestration** | 5 | Execution, cancellation, job management |
| **Execution Reporting** | 4 | Reports, logs, artifacts, diagnostics |
| **Analytics & Insights** | 5 | Dashboards, trends, flakiness, coverage, LLM costs |
| **Defect Management** | 6 | Create, update, link, sync to Jira/Azure/GitHub |
| **Webhooks** | 5 | Event subscriptions, delivery management |
| **Database Admin** | 1 | Alembic migrations (admin only, 2FA required) |

### Quick API Examples

### Core Workflow

#### 1. Create a Project (AUT)

```http
POST /api/v1/projects/
```

```json
{
  "name": "My Web App",
  "aut_base_url": "https://example.com",
  "default_framework": "playwright_python",
  "selector_preference": "role",
  "auth_hints": "Login with admin/admin on /login"
}
```

#### 2. Add a Test Case

```http
POST /api/v1/projects/{project_id}/test-cases/
```

```json
{
  "title": "TC-001 Successful Login",
  "format": "step_based",
  "content": "Step 1: Navigate to /login\nStep 2: Enter username 'admin'\nStep 3: Enter password 'admin123'\nStep 4: Click Login button\nExpected: User is redirected to /dashboard and sees 'Welcome' message",
  "preconditions": "User account exists in the system",
  "test_data_hints": "username=admin, password=admin123"
}
```

#### 3. Generate a Script

```http
POST /api/v1/projects/{project_id}/test-cases/{tc_id}/scripts/generate
```

```json
{
  "llm_provider": "openrouter",
  "llm_model": "openai/gpt-4o",
  "framework": "playwright_python"
}
```

Returns `202 Accepted` immediately. The agent runs in the background.

#### 4. Poll for the Result

```http
GET /api/v1/projects/{project_id}/test-cases/{tc_id}/scripts/{script_id}
```

Status values: `pending` → `generating` → `completed` | `failed`

#### 5. Run a Generated Playwright Script

```http
POST /api/v1/projects/{project_id}/test-cases/{tc_id}/scripts/{script_id}/run
```

Returns a persisted run record with:
- `status` (`completed` | `failed` | `timed_out`)
- `stdout`, `stderr`
- `exit_code`, `duration_seconds`

Execution safeguards:
- Script content is statically validated before execution.
- Unsafe imports and unsafe builtin calls are rejected and persisted as failed runs.
- Runtime uses Python isolated mode with a restricted environment.

#### 6. List Script Run History

```http
GET /api/v1/projects/{project_id}/test-cases/{tc_id}/scripts/{script_id}/runs
```

#### 7. Bulk Generate Scripts (Project-level)

```http
POST /api/v1/projects/{project_id}/scripts/bulk-generate
```

```json
{
  "llm_provider": "openrouter",
  "llm_model": "openai/gpt-4o",
  "framework": "playwright_python",
  "test_case_ids": [1, 2, 3]
}
```

#### 8. Bulk Run Latest Completed Scripts

```http
POST /api/v1/projects/{project_id}/scripts/bulk-run
```

#### 9. Poll Bulk Job Status

```http
GET /api/v1/projects/{project_id}/scripts/bulk-jobs/{job_id}
```

#### 10. Get Run Analytics (Project-level)

```http
GET /api/v1/projects/{project_id}/analytics/runs
```

Returns aggregate execution metrics and latest failure details.

---

## LangGraph Agent Pipeline

```
┌─────────────────┐     ┌──────────────────┐     ┌───────────────┐
│  parse_intent   │────▶│ generate_script  │────▶│ review_script │
│                 │     │                  │     │               │
│ Extracts:       │     │ Produces full    │     │ QA checks:    │
│ • Business goal │     │ framework-       │     │ • Assertions  │
│ • Actions list  │     │ specific script  │     │ • TODO markers│
│ • Assertions    │     │                  │     │ • Rewrites if │
│ • Preconditions │     │                  │     │   needed      │
└─────────────────┘     └──────────────────┘     └───────────────┘
```

---

## Supported Frameworks

| Key | Framework |
|---|---|
| `playwright_python` | Playwright for Python (default) |
| `selenium_python` | Selenium WebDriver Python |
| `uft_vbscript` | UFT / QTP VBScript |
| `cypress_js` | Cypress JavaScript |

---

## Project Structure

```
scriptgini/
├── app/
│   ├── main.py                   # FastAPI application
│   ├── config.py                 # Settings loaded from .env
│   ├── database.py               # SQLAlchemy engine + session
│   ├── models/
│   │   ├── project.py            # Project / AUT model
│   │   ├── test_case.py          # Test case model
│   │   └── generated_script.py  # Script history model
│   ├── schemas/                  # Pydantic request/response schemas
│   ├── routers/
│   │   ├── projects.py           # CRUD — projects
│   │   ├── test_cases.py         # CRUD — test cases
│   │   └── scripts.py            # Generate + history
│   ├── agents/
│   │   ├── script_gini_agent.py  # LangGraph graph definition
│   │   └── prompts.py            # All prompt templates
│   └── llm/
│       └── provider.py           # LLM provider factory
├── alembic/                      # Database migration scripts
├── alembic.ini
├── requirements.txt
├── .env.example                  # Template — copy to .env
├── start.bat                     # Windows launcher
└── start.sh                      # Linux / macOS launcher
```

---

## Database Migrations

Migrations are handled automatically by `start.bat` / `start.sh`.

To run manually:

```bash
# Apply all pending migrations
alembic upgrade head

# Create a new migration after model changes
alembic revision --autogenerate -m "description"

# Rollback one step
alembic downgrade -1
```

---

## Quality Gate Policy

Every check-in is expected to pass:

1. Unit tests
2. **100%** coverage on `app/`
3. `pip-audit`
4. Trivy filesystem scan

Local commands:

```bat
test.bat
audit.bat
trivy.bat
```

```bash
./test.sh
./audit.sh
./trivy.sh
```

A CI gate is configured in `.github/workflows/quality-gate.yml` to enforce the same checks on push/PR.

---

## Adding a New LLM Provider

1. Add config keys to `app/config.py`
2. Add a new `_provider()` function in `app/llm/provider.py`
3. Register it in `get_llm()` and the `LLMProvider` type alias
4. Add the corresponding key to `.env.example`

---

## Security Notes

- `.env` is git-ignored — never commit API keys
- The API has no authentication by default — add an API key middleware before exposing to a network
- UI validation only — the agent never makes live requests to the AUT

## Change Reports

- Commit-range report (`dfdf693b..15328b7`): `docs/changes-dfdf693b-to-15328b7.md`

---

## Development Roadmap

The project follows an **enterprise-grade development roadmap** with 6 sprints covering 190 story points over ~12 weeks. See [docs/todo.md](docs/todo.md) for detailed sprint breakdown with features, user stories, and tasks.

### Sprint Summary

| Sprint | Focus | Effort | Status |
|--------|-------|--------|--------|
| **Sprint 1** | IAM Core | 30-36pts | 🟡 Core delivered (auth hardening pending) |
| **Sprint 2** | RBAC + Multi-Tenancy | 32-38pts | 🟡 Core delivered (RBAC hardening pending) |
| **Sprint 3** | Durable Execution | 34-40pts | ✅ Completed (Redis + Celery queue foundation) |
| **Sprint 4** | Security & Hardening | 30-36pts | 🟡 In progress (isolation boundary + breakout tests pending) |
| **Sprint 5** | Reporting & Analytics | 28-34pts | ✅ Completed (Reports APIs, trends/flakiness, retention cleanup) |
| **Sprint 6** | Advanced Features | 24-30pts | 🔲 Pending (Webhooks, defect sync, versioning) |

---

## License

MIT
