Metadata-Version: 2.4
Name: scriptgini
Version: 1.7.6
Summary: Agentic AI system that converts functional test cases into automation test scripts.
Author: ScriptGini Team
License: Proprietary
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Framework :: FastAPI
Requires-Python: >=3.11
Description-Content-Type: text/markdown

# ScriptGini

> **Enterprise-grade Agentic AI system that converts functional test cases into high-quality, review-ready automation test scripts.**

Current release: v1.7.6 (Patch release - Restores 100% coverage by closing cache/provider branch gaps)

Latest validated status (2026-05-12):
- Test run: 286 passed, 4 deselected
- Coverage: 100% on `app/` (2952/2952 statements)
- PostgreSQL integration suite: opt-in and skip-by-default unless explicitly enabled

---

## What is ScriptGini?

ScriptGini is an AI-powered test automation engine built for Quality Engineering teams. You feed it a functional test case and an Application Under Test (AUT) URL â€” it returns a production-ready automation script in your chosen framework, generated by a multi-step LangGraph agent that reasons about test intent before writing a single line of code.

---

## Features

- **Agentic 3-node LangGraph pipeline** â€” Intent analysis â†’ Script generation â†’ Quality review
- **Multi-provider LLM support** â€” OpenAI, Ollama (local), OpenRouter, Google Gemini, AWS Bedrock
- **Framework-agnostic output** â€” Playwright Python, Selenium Python, UFT VBScript, Cypress JS
- **Intelligent selector strategy** â€” Role â†’ Label â†’ data-testid â†’ CSS â†’ XPath (last resort)
- **Project & AUT management** â€” Store multiple projects, each with its own base URL and defaults
- **Full test case history** â€” Every generated script is stored in SQLite with status and token usage
- **Execution history persistence** â€” Every run is stored in `script_runs` with stdout/stderr, exit code, and duration
- **Durable execution APIs** â€” Async execution jobs with run/status/abort endpoints and idempotency-key support
- **Hardened execution sandbox** â€” Script runs use isolated Python mode, static safety validation, and restricted environment variables
- **Bulk job orchestration** â€” Project-level bulk generate and bulk run with pollable job status
- **Run analytics dashboard** â€” Project-level pass/fail/timeout metrics and recent failure feed
- **Coverage analytics** â€” Module-level coverage and execution visibility for test cases
- **Richer test case intake** â€” Import `.txt`, `.md`, `.json`, `.csv`, `.feature`, `.yml/.yaml`, and `.xlsx`
- **Import preview mapping** â€” Preview parsed scenarios in the UI before creating a project workspace
- **Script lifecycle tracking** â€” Refactor, version history, diff, and rollback support
- **REST API** â€” FastAPI with auto-generated Swagger UI
- **Alembic migrations** â€” Safe, versioned schema management over SQLite

---

## Tech Stack

| Layer | Technology |
|---|---|
| API | FastAPI + Uvicorn |
| Agentic AI | LangGraph + LangChain |
| LLM Providers | OpenAI, Ollama, OpenRouter, Gemini, Bedrock |
| Database | SQLite |
| ORM | SQLAlchemy 2.0 |
| Migrations | Alembic |
| Config | Pydantic Settings (.env) |

---

## Quick Start

### Windows

```bat
start.bat
```

### Linux / macOS

```bash
chmod +x start.sh
./start.sh
```

The script will:
1. Create a Python virtual environment
2. Install all dependencies
3. Copy `.env.example` â†’ `.env` if missing (edit it before re-running)
4. Run Alembic migrations
5. Start the server and open Swagger UI in your browser

To load a ready-made sample workspace, use the `Load Demo Project` button in the web UI or call `POST /api/v1/demo/load`.

---

## Configuration

Copy `.env.example` to `.env` and fill in the values you need:

```bash
cp .env.example .env
```

```env
# Choose your default provider
DEFAULT_LLM_PROVIDER=openrouter   # openai | ollama | openrouter | gemini | bedrock

# OpenAI
OPENAI_API_KEY=your_openai_api_key_here

# Ollama (local â€” no key needed)
OLLAMA_BASE_URL=http://localhost:11434
OLLAMA_MODEL=llama3
OLLAMA_NUM_PREDICT=700

# Generation latency controls
LLM_REQUEST_TIMEOUT_SECONDS=45
SCRIPT_GENERATION_TIMEOUT_SECONDS=180
SKIP_REVIEW_FOR_OLLAMA=true

# OpenRouter
OPENROUTER_API_KEY=your_openrouter_api_key_here
OPENROUTER_MODEL=openai/gpt-4o

# Google Gemini
GOOGLE_API_KEY=your_google_api_key_here

# AWS Bedrock
AWS_ACCESS_KEY_ID=your_aws_access_key_id
AWS_SECRET_ACCESS_KEY=your_aws_secret_access_key
AWS_REGION_NAME=us-east-1
```

> `.env` is git-ignored. Never commit real API keys.

If local generation feels slow, reduce `OLLAMA_NUM_PREDICT`, keep `SKIP_REVIEW_FOR_OLLAMA=true`, or switch to a smaller/faster Ollama model.

---

## OpenAPI Specification

**ScriptGini implements a production-ready OpenAPI 3.0.3 specification** that defines a complete enterprise REST API with 50+ endpoints, 60+ schemas, and comprehensive documentation.

### Specification Highlights

**The API specification includes 12 salient features**:

1. **Multi-tenant Architecture** â€” Organizations, Workspaces, Teams with hierarchical RBAC
2. **Advanced LLM Management** â€” Multi-provider orchestration, health monitoring, cost tracking, and model governance
3. **Intelligent Test Data Management** â€” Multi-source ingestion, synthetic generation, PII masking, state locking, placeholder mapping
4. **Asynchronous Job Management** â€” HTTP 202 Accepted, idempotent execution, job polling, webhooks
5. **Comprehensive Reporting** â€” Structured logs, artifact storage, signed downloads, execution diagnostics
6. **Analytics & Insights** â€” Dashboards, trend analysis, flakiness detection, code coverage, LLM usage
7. **Defect Lifecycle** â€” Auto-detection, Jira/Azure/GitHub sync, severity tracking, traceability
8. **Script Versioning** â€” Git-style history, quality metrics, refactoring, diff metadata
9. **Webhooks** â€” Event-driven notifications, delivery guarantees, retry policies
10. **Security & Authorization** â€” JWT + OAuth2, API keys with scopes, RBAC enforcement, rate limiting
11. **Error Handling & Observability** â€” Standardized error envelope, request ID tracing, correlation IDs
12. **Pagination & Filtering** â€” limit/offset/sort, comprehensive filtering, faceted search

### API Reference

Once running, visit:

| URL | Description |
|---|---|
| `http://localhost:8000/docs` | Swagger UI (interactive) |
| `http://localhost:8000/redoc` | ReDoc |
| `http://localhost:8000/health` | Health check |
| `http://localhost:8000/openapi.json` | Raw OpenAPI 3.0.3 specification |

### Full OpenAPI Documentation

- **Complete Specification**: [docs/openapi-improved-draft-2026-05-10.yaml](docs/openapi-improved-draft-2026-05-10.yaml)
- **Feature Documentation**: [docs/OPENAPI-SPECIFICATION.md](docs/OPENAPI-SPECIFICATION.md) â€” Covers all 12 salient features with examples and flows
- **API Architecture Review**: [docs/api-architecture-review-2026-05-10.md](docs/api-architecture-review-2026-05-10.md) â€” Analysis of current state vs. target enterprise API

### API Domains (50+ Endpoints)

| Domain | Endpoints | Purpose |
|--------|-----------|---------|
| **Authentication & IAM** | 8 | Login, token refresh, user management, API keys |
| **Organizations & Teams** | 6 | Multi-tenancy, team management, RBAC |
| **Workspaces & Projects** | 5 | Organizational hierarchy, configuration |
| **Test Data Management** | 11 | Data sets, reservations, masking, synthetic generation |
| **LLM Management** | 5 | Provider registration, model governance, cost tracking |
| **Script Engineering** | 7 | Generation, versioning, quality, refactoring |
| **Test Cases** | 4 | CRUD operations for test definitions |
| **Test Orchestration** | 5 | Execution, cancellation, job management |
| **Execution Reporting** | 4 | Reports, logs, artifacts, diagnostics |
| **Analytics & Insights** | 5 | Dashboards, trends, flakiness, coverage, LLM costs |
| **Defect Management** | 6 | Create, update, link, sync to Jira/Azure/GitHub |
| **Webhooks** | 5 | Event subscriptions, delivery management |
| **Database Admin** | 1 | Alembic migrations (admin only, 2FA required) |

### Quick API Examples

### Core Workflow

#### 1. Create a Project (AUT)

```http
POST /api/v1/projects/
```

```json
{
  "name": "My Web App",
  "aut_base_url": "https://example.com",
  "default_framework": "playwright_python",
  "selector_preference": "role",
  "auth_hints": "Login with admin/admin on /login"
}
```

#### 2. Add a Test Case

```http
POST /api/v1/projects/{project_id}/test-cases/
```

```json
{
  "title": "TC-001 Successful Login",
  "format": "step_based",
  "content": "Step 1: Navigate to /login\nStep 2: Enter username 'admin'\nStep 3: Enter password 'admin123'\nStep 4: Click Login button\nExpected: User is redirected to /dashboard and sees 'Welcome' message",
  "preconditions": "User account exists in the system",
  "test_data_hints": "username=admin, password=admin123"
}
```

#### 3. Generate a Script

```http
POST /api/v1/projects/{project_id}/test-cases/{tc_id}/scripts/generate
```

```json
{
  "llm_provider": "openrouter",
  "llm_model": "openai/gpt-4o",
  "framework": "playwright_python"
}
```

Returns `202 Accepted` immediately. The agent runs in the background.

#### 4. Poll for the Result

```http
GET /api/v1/projects/{project_id}/test-cases/{tc_id}/scripts/{script_id}
```

Status values: `pending` â†’ `generating` â†’ `completed` | `failed`

#### 5. Run a Generated Playwright Script

```http
POST /api/v1/projects/{project_id}/test-cases/{tc_id}/scripts/{script_id}/run
```

Returns a persisted run record with:
- `status` (`completed` | `failed` | `timed_out`)
- `stdout`, `stderr`
- `exit_code`, `duration_seconds`

Execution safeguards:
- Script content is statically validated before execution.
- Unsafe imports and unsafe builtin calls are rejected and persisted as failed runs.
- Runtime uses Python isolated mode with a restricted environment.

#### 6. List Script Run History

```http
GET /api/v1/projects/{project_id}/test-cases/{tc_id}/scripts/{script_id}/runs
```

#### 7. Bulk Generate Scripts (Project-level)

```http
POST /api/v1/projects/{project_id}/scripts/bulk-generate
```

```json
{
  "llm_provider": "openrouter",
  "llm_model": "openai/gpt-4o",
  "framework": "playwright_python",
  "test_case_ids": [1, 2, 3]
}
```

#### 8. Bulk Run Latest Completed Scripts

```http
POST /api/v1/projects/{project_id}/scripts/bulk-run
```

#### 9. Poll Bulk Job Status

```http
GET /api/v1/projects/{project_id}/scripts/bulk-jobs/{job_id}
```

#### 10. Get Run Analytics (Project-level)

```http
GET /api/v1/projects/{project_id}/analytics/runs
```

Returns aggregate execution metrics and latest failure details.

---

## LangGraph Agent Pipeline

```
â”Œâ”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”     â”Œâ”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”     â”Œâ”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”
â”‚  parse_intent   â”‚â”€â”€â”€â”€â–¶â”‚ generate_script  â”‚â”€â”€â”€â”€â–¶â”‚ review_script â”‚
â”‚                 â”‚     â”‚                  â”‚     â”‚               â”‚
â”‚ Extracts:       â”‚     â”‚ Produces full    â”‚     â”‚ QA checks:    â”‚
â”‚ â€¢ Business goal â”‚     â”‚ framework-       â”‚     â”‚ â€¢ Assertions  â”‚
â”‚ â€¢ Actions list  â”‚     â”‚ specific script  â”‚     â”‚ â€¢ TODO markersâ”‚
â”‚ â€¢ Assertions    â”‚     â”‚                  â”‚     â”‚ â€¢ Rewrites if â”‚
â”‚ â€¢ Preconditions â”‚     â”‚                  â”‚     â”‚   needed      â”‚
â””â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”˜     â””â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”˜     â””â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”˜
```

---

## Supported Frameworks

| Key | Framework |
|---|---|
| `playwright_python` | Playwright for Python (default) |
| `selenium_python` | Selenium WebDriver Python |
| `uft_vbscript` | UFT / QTP VBScript |
| `cypress_js` | Cypress JavaScript |

---

## Project Structure

```
scriptgini/
â”œâ”€â”€ app/
â”‚   â”œâ”€â”€ main.py                   # FastAPI application
â”‚   â”œâ”€â”€ config.py                 # Settings loaded from .env
â”‚   â”œâ”€â”€ database.py               # SQLAlchemy engine + session
â”‚   â”œâ”€â”€ models/
â”‚   â”‚   â”œâ”€â”€ project.py            # Project / AUT model
â”‚   â”‚   â”œâ”€â”€ test_case.py          # Test case model
â”‚   â”‚   â””â”€â”€ generated_script.py  # Script history model
â”‚   â”œâ”€â”€ schemas/                  # Pydantic request/response schemas
â”‚   â”œâ”€â”€ routers/
â”‚   â”‚   â”œâ”€â”€ projects.py           # CRUD â€” projects
â”‚   â”‚   â”œâ”€â”€ test_cases.py         # CRUD â€” test cases
â”‚   â”‚   â””â”€â”€ scripts.py            # Generate + history
â”‚   â”œâ”€â”€ agents/
â”‚   â”‚   â”œâ”€â”€ script_gini_agent.py  # LangGraph graph definition
â”‚   â”‚   â””â”€â”€ prompts.py            # All prompt templates
â”‚   â””â”€â”€ llm/
â”‚       â””â”€â”€ provider.py           # LLM provider factory
â”œâ”€â”€ alembic/                      # Database migration scripts
â”œâ”€â”€ alembic.ini
â”œâ”€â”€ requirements.txt
â”œâ”€â”€ .env.example                  # Template â€” copy to .env
â”œâ”€â”€ start.bat                     # Windows launcher
â””â”€â”€ start.sh                      # Linux / macOS launcher
```

---

## Database Migrations

Migrations are handled automatically by `start.bat` / `start.sh`.

To run manually:

```bash
# Apply all pending migrations
alembic upgrade head

# Create a new migration after model changes
alembic revision --autogenerate -m "description"

# Rollback one step
alembic downgrade -1
```

---

## Quality Gate Policy

Every check-in is expected to pass:

1. Unit tests
2. **100%** coverage on `app/`
3. `pip-audit`
4. Trivy filesystem scan

Local commands:

```bat
test.bat
audit.bat
trivy.bat
```

Opt-in PostgreSQL integration round-trip tests:

```bat
set RUN_POSTGRES_INTEGRATION_TESTS=1
set INTEGRATION_DATABASE_URL=postgresql://scriptgini:scriptgini@localhost:5433/scriptgini_test_db
test_integration_postgres.bat
```

```bash
./test.sh
./audit.sh
./trivy.sh
```

```bash
export RUN_POSTGRES_INTEGRATION_TESTS=1
export INTEGRATION_DATABASE_URL=postgresql://scriptgini:scriptgini@localhost:5433/scriptgini_test_db
./test_integration_postgres.sh
```

Note: Integration tests are intentionally skip-by-default and include a safety check to avoid running against non-test databases.

A CI gate is configured in `.github/workflows/quality-gate.yml` to enforce the same checks on push/PR.

---

## Adding a New LLM Provider

1. Add config keys to `app/config.py`
2. Add a new `_provider()` function in `app/llm/provider.py`
3. Register it in `get_llm()` and the `LLMProvider` type alias
4. Add the corresponding key to `.env.example`

---

## Security Notes

ScriptGini ships with a layered security model that is active in the default configuration:

### Authentication & Authorization
- **JWT Bearer tokens** (access + refresh, configurable expiry) protect all non-public endpoints
- **API keys** (hashed storage, named scopes, revocable) enable CI/CD pipelines without sharing user credentials
- **RBAC** â€” four project roles (owner / admin / member / viewer) control read and write access per project
- All auth events (login, register, API-key create/revoke, forbidden access) are written to structured security audit logs

### Network Controls
- **CORS** â€” only origins listed in `CORS_ALLOWED_ORIGINS` (env var) are permitted; wildcard `*` is never set in production configs
- **Rate limiting** â€” two layers:
  - *Global middleware*: 120 req / 60 s per IP for all non-exempt paths (configurable)
  - *Per-user/API-key*: tighter limits on sensitive actions (login: 10/min, register: 5/min, execution: 20/min, bulk jobs: 10/min)
  - Standard `X-RateLimit-*` and `Retry-After` headers are included on all responses

### Execution Isolation
- Scripts run inside a **constrained runner** â€” only an explicit allowlist of safe import roots is permitted
- Runtime attribute calls are blocked (`system`, `popen`, `connect`, etc.)
- `__builtins__`, `__globals__`, and introspection symbols are blocked by name
- Execution environment variables are narrowed to an explicit `EXECUTION_ENV_ALLOWED_KEYS` allowlist
- Hard + soft execution timeouts enforce maximum run duration

### Secrets & Config
- `.env` is git-ignored â€” never commit API keys or `JWT_SECRET_KEY`
- All secrets are consumed via `pydantic-settings` env-var injection; no hardcoded defaults are safe for production
- `JWT_SECRET_KEY` **must** be replaced with a cryptographically random value before deployment
- The UI performs client-side validation only; the agent never makes unsolicited live requests to the AUT

## Change Reports

- Commit-range report (`dfdf693b..15328b7`): `docs/changes-dfdf693b-to-15328b7.md`

---

## Development Roadmap

The project follows an **enterprise-grade development roadmap** with 6 sprints covering 190 story points over ~12 weeks. See [docs/todo.md](docs/todo.md) for detailed sprint breakdown with features, user stories, and tasks.

### Sprint Summary

| Sprint | Focus | Effort | Status |
|--------|-------|--------|--------|
| **Sprint 1** | IAM Core | 30-36pts | ðŸŸ¡ Core delivered (auth hardening pending) |
| **Sprint 2** | RBAC + Multi-Tenancy | 32-38pts | ðŸŸ¡ Core delivered (RBAC hardening pending) |
| **Sprint 3** | Durable Execution | 34-40pts | âœ… Completed (Redis + Celery queue foundation) |
| **Sprint 4** | Security & Hardening | 30-36pts | âœ… Hardening increment completed (isolation + negative tests + audit controls) |
| **Sprint 5** | Reporting & Analytics | 28-34pts | âœ… Completed (Reports APIs, trends/flakiness, retention cleanup) |
| **Sprint 6** | Coverage Analytics, Script Lifecycle, and DX | 24-30pts | âœ… Completed (coverage analytics, refactor/version history/diff/rollback, 100% quality gate) |
| **Sprint 7** | Hardening Completion + Developer Experience | 20-24pts | âœ… Completed (per-user rate limiting, analytics indexes, OpenAPI scope docs, 286 passed with integration tests deselected by default, 100% coverage) |

---

## License

MIT

