Metadata-Version: 2.4
Name: synthesize-ai
Version: 0.1.1
Summary: Natural language testing platform powered by AI — transform plain English into executable tests across any framework
License: MIT
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: anthropic>=0.40.0
Requires-Dist: robotframework>=7.0
Requires-Dist: click>=8.0
Requires-Dist: pydantic>=2.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: jinja2>=3.1
Requires-Dist: rich>=13.0
Requires-Dist: chromadb>=0.5.0
Requires-Dist: sentence-transformers>=3.0
Requires-Dist: gitpython>=3.1.0
Provides-Extra: web
Requires-Dist: robotframework-seleniumlibrary>=6.0; extra == "web"
Provides-Extra: api
Requires-Dist: robotframework-requests>=0.9; extra == "api"
Provides-Extra: db
Requires-Dist: robotframework-databaselibrary>=2.0; extra == "db"
Provides-Extra: watch
Requires-Dist: watchdog>=4.0; extra == "watch"
Provides-Extra: visual
Requires-Dist: Pillow>=10.0; extra == "visual"
Provides-Extra: docker
Provides-Extra: all
Requires-Dist: synthesize-ai[api,db,docker,visual,watch,web]; extra == "all"
Provides-Extra: dashboard
Requires-Dist: flask>=3.0; extra == "dashboard"
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: pytest-cov>=5.0; extra == "dev"
Requires-Dist: pytest-mock>=3.12; extra == "dev"
Requires-Dist: pytest-agents>=1.0.10; extra == "dev"
Requires-Dist: log-sculptor>=0.1.0; extra == "dev"
Requires-Dist: watchdog>=4.0; extra == "dev"
Requires-Dist: Pillow>=10.0; extra == "dev"
Dynamic: license-file

# Synthesize

**Natural language testing platform powered by AI**

Stop writing test code. Start describing what you want tested.

```
English in → Tests out → Results back
```

[![Tests](https://img.shields.io/badge/tests-2162%20passed-brightgreen)](https://synthesize-ai.dev)
[![Plugins](https://img.shields.io/badge/plugins-6%20frameworks-blue)](https://synthesize-ai.dev/plugins)
[![License](https://img.shields.io/badge/license-MIT-blue)](LICENSE)
[![Python](https://img.shields.io/badge/python-3.9%2B-blue)](https://www.python.org/)

[Website](https://synthesize-ai.dev) • [Documentation](https://docs.synthesize-ai.dev) • [Plugin Registry](https://plugins.synthesize-ai.dev) • [Discord](https://discord.gg/synthesize)

---

## What is Synthesize?

Synthesize transforms plain English descriptions into executable tests across **any testing framework**. No Gherkin, no keywords, no boilerplate—just describe what you want tested.

**Write this:**
```yaml
suite: User Authentication
tests:
  - User can login with valid credentials
  - Bad password shows error message
  - Account locks after 5 failed attempts
```

**Get working tests in:**
- Robot Framework
- Playwright
- Cypress
- Selenium
- Cucumber
- pytest

**With built-in:**
- ✅ Self-healing when tests break
- ✅ Compliance traceability (SOC2, HIPAA, SOX)
- ✅ Cost optimization (70%+ reduction in LLM costs)
- ✅ Historical analytics and flakiness detection
- ✅ Team collaboration and knowledge sharing

---

## Why Synthesize?

### For QA Engineers
**Stop maintaining brittle tests**
- Self-healing automatically fixes broken selectors
- Multi-framework support—use what you know
- 70% less time on test maintenance

### For Engineering Teams
**Ship faster with confidence**
- Write tests in English, not code
- Parallel execution and smart test selection
- Real-time collaboration and reviews

### For Enterprises
**Meet compliance requirements effortlessly**
- Built-in SOC2, HIPAA, SOX mapping
- Audit trails and traceability reports
- Self-hosted deployment option

---

## Quick Start

### Installation

```bash
# Install Synthesize
pip install synthesize-ai

# Set your API key
export ANTHROPIC_API_KEY=sk-ant-...

# Initialize a project
synthesize init
```

### Your First Test

Create a test file (`tests/login.synth`):

```yaml
suite: User Login
context: Web app, login page at /auth/login

tests:
  - User can login with valid credentials
  - User sees error with invalid password
  - Session expires after 30 minutes
```

Generate and run:

```bash
# Generate Robot Framework tests
synthesize generate tests/login.synth

# Or generate Playwright
synthesize generate tests/login.synth --format playwright

# Execute with self-healing
synthesize run tests/login.synth --heal auto --max-retries 3
```

---

## Features

### 🤖 AI-Powered Generation
- **Natural Language Processing**: Claude interprets your intent
- **Context-Aware**: Understands your application's domain
- **Multi-Framework**: Same intent → 6 different frameworks

### 🔧 Self-Healing Tests
- **Automatic Repair**: Fixes broken locators and assertions
- **Smart Retry**: Learns from failures
- **Confidence Scoring**: Reports healing reliability

### 📊 Analytics & Insights
- **Flakiness Detection**: Identifies unstable tests automatically
- **Trend Analysis**: Track test duration and failure patterns
- **Predictive**: ML-powered failure prediction

### 🔒 Security & Compliance
- **OWASP ZAP Integration**: Dynamic security scanning
- **Secrets Detection**: Prevent credential leaks
- **Compliance Mapping**: SOC2, HIPAA, SOX traceability

### 💰 Cost Optimization
- **LLM Caching**: 60-70% cache hit rate
- **Local Models**: Ollama support for simple tasks
- **Smart Routing**: Haiku for simple, Opus for complex
- **Result**: 70%+ cost reduction

### 👥 Team Collaboration
- **Team Project Sharing**: Team members see and edit shared projects
- **Team Secrets Vault**: Shared encrypted secrets injected at runtime
- **Admin User Management**: Create/manage users with support for email, Google, GitHub, SAML, LDAP auth
- **Email Invites**: Team invitations via Amazon SES (DKIM-verified domain)
- **Role-Based Access**: Admin, tester, and user roles with plan-based feature gating

---

## Supported Frameworks

| Framework | Status | Plugin |
|-----------|--------|--------|
| Robot Framework | ✅ Built-in | `robot` |
| Playwright | ✅ Built-in | `playwright` |
| Cypress | ✅ Built-in | `cypress` |
| Selenium | ✅ Built-in | `selenium` |
| Cucumber/Behave | ✅ Built-in | `cucumber` |
| pytest | ✅ Built-in | `pytest-agents` |
| TestCafe | 🔌 Plugin Available | `testcafe` |
| Puppeteer | 🔌 Plugin Available | `puppeteer` |
| Appium | 🔌 Community | `appium` |
| Custom | 🛠️ Build Your Own | [Guide](https://docs.synthesize-ai.dev/plugins) |

---

## Examples

### Web Testing
```yaml
suite: E-commerce Checkout
context: Online store, products at /products

tests:
  - User can add item to cart
  - User can apply discount code
  - User can complete checkout with credit card
  - User receives order confirmation email
```

### API Testing
```yaml
suite: User API
context: REST API at https://api.example.com
environment:
  type: api
  base_url: ${API_URL}

tests:
  - GET /users returns 200 with user list
  - POST /users creates new user
  - PUT /users/:id updates existing user
  - DELETE /users/:id returns 404 for non-existent user
```

### Database Testing
```yaml
suite: Data Migration Validation
context: MySQL database, compliance_records table
compliance: [HIPAA-164.312, SOX-302]

tests:
  - Record count matches source
  - No orphaned junction records
  - PII fields are encrypted at rest
```

### Compliance Testing
```yaml
suite: HIPAA Compliance Validation
context: Patient portal, PHI data handling
compliance: [HIPAA-164.312(a)(1), HIPAA-164.312(e)(1)]

tests:
  - PHI is encrypted in transit
  - Access logs record all PHI access
  - Session timeout enforces 15-minute limit
  - Audit trail captures all data modifications
```

---

## CLI Reference

### Test Generation & Execution
```bash
synthesize generate <test.synth>              # Generate .robot files
synthesize generate <test.synth> --format <format>  # Choose framework
synthesize run <test.synth>                   # Generate + execute
synthesize run <test.synth> --heal auto       # With self-healing
synthesize run <test.synth> --smart           # Smart test selection
```

### Plugin Management
```bash
synthesize plugin list                       # Show installed plugins
synthesize plugin install <name>             # Install plugin
synthesize plugin create <name>              # Create new plugin
synthesize plugin formats                    # List available formats
```

### Cost & Analytics
```bash
synthesize cost report                       # LLM cost analysis
synthesize cost stats                        # Cache statistics
synthesize analytics trend <test>            # Duration trends
synthesize analytics flaky-tests             # Identify flaky tests
```

### Team Collaboration
```bash
synthesize sync                              # Sync shared knowledge
synthesize review submit <test.synth>         # Submit for review
synthesize review list --assigned-to-me      # My reviews
```

### Security
```bash
synthesize security scan <test.synth>         # Security scan
synthesize security scan-secrets ./          # Detect leaked secrets
```

[Full CLI Reference](https://docs.synthesize-ai.dev/cli)

---

## Tutorials

The [Developer Guide](docs/DEVELOPER_GUIDE.md) contains 12 step-by-step tutorials:

1. **Your First Test** — Create and run an AI-generated test in 2 minutes
2. **API Testing from OpenAPI** — Auto-generate tests from Swagger/Postman/DDL specs
3. **Self-Healing Tests** — Auto-fix broken locators and assertions
4. **Compliance Auditing** — SOC2/HIPAA/SOX mapping and audit reports
5. **Watch Mode for TDD** — Live-reload pipeline on `.synth` file changes
6. **Performance Testing** — HTTP load tests with baseline regression detection
7. **CI/CD Integration** — GitHub Actions, GitLab CI, Jenkins configs
8. **Import Existing Tests** — Migrate `.robot` and `.feature` files to `.synth`
9. **Interactive Chat Mode** — Conversational test authoring
10. **Multi-Environment Promotion** — Gate-based dev → staging → production pipeline
11. **Source Code Analysis** — Scan React/Vue/Django/Flask apps, generate BDD tests
12. **LLM Cost Optimization** — Caching, local models, smart routing

---

## Synthesize Studio

**Web-based multi-framework testing platform**

- 🌐 **Live at**: https://studio.synthesize-ai.dev
- 🎨 **Monaco Editor**: VS Code-like editing experience
- ⚡ **Real-time Execution**: WebSocket-powered live run progress
- 🔄 **Self-Healing UI**: Visual healing suggestions with Heal All for batch fixes
- 📊 **Dashboard**: Recharts-powered stats, trends, and recent runs with drill-down
- 💰 **Cost Monitor**: Per-user LLM cost tracking with daily/provider/task breakdowns
- 💬 **Framework-Aware Chat**: AI assistant adapts to your selected framework
- 📄 **Report Viewer**: Inline report.html and log.html with embedded screenshots
- 🔍 **Page Analyzer**: Extract DOM elements and drag-and-drop into .synth editor
- 📈 **Analytics**: Trend analysis, flaky test detection, regression detection, ML failure predictions
- 🔒 **Security Center**: Secrets scanning, risk scoring, OWASP ZAP integration
- 👥 **Team Sharing**: Create teams, invite members via email (SES), share projects
- 📤 **GitHub Export**: Push projects directly to GitHub repos (paid plans)
- 📚 **Tutorials**: 12 step-by-step in-app tutorials from beginner to advanced
- 🔌 **21 Lifecycle Hooks**: Extensible hook system for framework plugins
- 🔐 **Auth**: Local email/password, GitHub OAuth, Google OAuth with plan-based feature gating
- 🔑 **Secrets Vault**: Personal and team-scoped encrypted secrets, injected into test generation and execution
- 🛡️ **Admin Panel**: User CRUD, role/plan management, activity monitoring, admin-provisioned accounts (email, SAML, LDAP)

**Technology Stack:**
- Frontend: Next.js 14 + TypeScript + Tailwind + Zustand + Recharts
- Backend: FastAPI + PostgreSQL (SQLAlchemy async) + Anthropic Claude
- Email: Amazon SES API (HTTPS, DKIM-verified domain)
- Billing: Stripe (free/pro/team plans)
- Deployment: DigitalOcean DOKS (Kubernetes) with nginx ingress

[Try Studio →](https://studio.synthesize-ai.dev)

---

## VS Code Extension

**Rich IDE integration for `.synth` files**

- 🎨 Syntax highlighting
- ✅ Real-time validation
- 💡 IntelliSense for keywords and compliance tags
- 🔍 Hover documentation
- ⚡ CodeLens actions (Generate, Run, Preview)
- ⌨️ Keyboard shortcuts

**Install:**
```bash
code --install-extension synthesize.vscode-synthesize
```

[Extension Documentation →](https://docs.synthesize-ai.dev/vscode)

---

## Plugin System

Synthesize has a powerful plugin architecture with **21 lifecycle hooks**:

### Create a Plugin

```bash
# Scaffold a new plugin
synthesize plugin create my-framework

# Implement generator
cd ~/.synth/plugins/my-framework/
# Edit generator.py
```

**Minimal Plugin:**
```python
from synthesize.plugins.base import BaseGenerator

class MyFrameworkGenerator(BaseGenerator):
    def generate(self, intent):
        # Transform intent to test code
        return generated_code
    
    def get_file_extension(self):
        return '.test.js'
```

**Plugin Manifest (`plugin.yaml`):**
```yaml
name: my-framework
version: 1.0.0
provides:
  generators: [my-framework]
entry_points:
  generator: generator.MyFrameworkGenerator
```

### Available Hooks

Plugins can hook into 21 lifecycle events:

- `pre_generate`, `post_generate` - Code generation
- `pre_execute`, `post_execute` - Test execution
- `pre_heal`, `post_heal` - Self-healing
- `pre_validate`, `post_validate` - Test validation
- `pre_parse`, `post_parse` - Input parsing
- `pre_report`, `post_report` - Report generation
- `pre_analyze`, `post_analyze` - Analytics processing
- `pre_security_scan`, `post_security_scan` - Security scanning
- `pre_save`, `post_save` - Project saving
- `on_error`, `on_retry` - Error handling
- `on_test_pass`, `on_test_fail` - Test result callbacks

[Plugin Development Guide →](https://docs.synthesize-ai.dev/plugins)

---

## Architecture

```
┌─── Natural Language (.synth files) ──────────────┐
│  suite: Login Tests                             │
│  tests:                                          │
│    - User can login with valid credentials      │
└────────────────┬────────────────────────────────┘
                 ▼
┌─── AI Interpretation Engine ────────────────────┐
│  • Intent parsing (Claude)                      │
│  • Context understanding                        │
│  • Action decomposition                         │
│  • Data requirement extraction                  │
└────────────────┬────────────────────────────────┘
                 ▼
┌─── Code Generation (Multi-Framework) ───────────┐
│  • Plugin-based generators                      │
│  • Template rendering                           │
│  • Locator strategies                           │
└────────────────┬────────────────────────────────┘
                 ▼
┌─── Quality Gate ─────────────────────────────────┐
│  • Structural validation                        │
│  • Security checks                              │
│  • Compliance mapping                           │
└────────────────┬────────────────────────────────┘
                 ▼
┌─── Execution Engine ─────────────────────────────┐
│  • Smart orchestration                          │
│  • Parallel execution                           │
│  • Real-time monitoring                         │
└────────────────┬────────────────────────────────┘
                 ▼
         ┌───────┴───────┐
         │               │
       Pass            Fail
         │               │
         ▼               ▼
    ┌────────┐    ┌─────────────┐
    │Results │    │Self-Healing │
    │   DB   │    │   Engine    │
    └────┬───┘    └──────┬──────┘
         │               │
         └───────┬───────┘
                 ▼
┌─── Analytics & Insights ─────────────────────────┐
│  • Flakiness detection                          │
│  • Trend analysis                               │
│  • Cost tracking                                │
│  • Compliance reporting                         │
└──────────────────────────────────────────────────┘
```

---

## Configuration

**Global config (`.synth.yaml`):**

```yaml
# LLM Settings
llm:
  provider: anthropic
  model: claude-sonnet-4-5-20250929
  max_tokens: 4000
  
  # Cost Optimization
  cache:
    enabled: true
    ttl_seconds: 86400
  
  routing:
    simple: ollama/llama3.2    # Local model for simple tasks
    complex: anthropic/opus    # Claude Opus for complex

# Framework Plugins
plugins:
  enabled:
    - robot
    - playwright
    - cypress
  
  settings:
    robot:
      browser: chrome
      headless: false

# Self-Healing
healing:
  enabled: true
  max_retries: 3
  strategies:
    - locator_alternatives
    - ai_suggestion
    - dom_analysis

# Team Collaboration
collaboration:
  server_url: https://kb.synthesize-ai.dev
  team_id: ${TEAM_ID}
  auto_sync: true

# Security
security:
  zap:
    enabled: true
    active_scan: false
  secrets:
    enabled: true
    fail_on_high: true

# Analytics
analytics:
  enabled: true
  flake_detection: true
  trend_analysis: true
```

[Full Configuration Guide →](https://docs.synthesize-ai.dev/config)

---

## Pricing

### Open Source (Free Forever)
- ✅ Core CLI with all commands
- ✅ 6 built-in framework plugins
- ✅ Self-healing engine
- ✅ Local execution
- ✅ Community support

### Pro ($49/user/month)
- ✅ Everything in Open Source, plus:
- ✅ Synthesize Studio web application
- ✅ Cost optimization (saves $70+/month on LLM costs)
- ✅ Historical analytics and insights
- ✅ Priority support
- ✅ Cloud execution option

### Enterprise (Custom Pricing)
- ✅ Everything in Pro, plus:
- ✅ Team collaboration (shared projects, team secrets vault)
- ✅ Admin user provisioning (email, Google, GitHub, SAML, LDAP)
- ✅ Compliance reporting (SOC2, HIPAA, SOX)
- ✅ Security scanning and secrets detection
- ✅ Multi-environment promotion pipelines
- ✅ Self-hosted / private cloud deployment
- ✅ Satellite agents for firewall-restricted networks
- ✅ Custom plugins and integrations
- ✅ SLA and dedicated support

[View Pricing Details →](https://synthesize-ai.dev/pricing)

---

## Comparison

| Feature | Synthesize | Selenium Grid | Cypress Cloud | TestRail |
|---------|-----------|---------------|---------------|----------|
| Natural Language Tests | ✅ | ❌ | ❌ | ❌ |
| AI-Powered Generation | ✅ | ❌ | ❌ | ❌ |
| Self-Healing | ✅ | ❌ | ❌ | ❌ |
| Multi-Framework | ✅ (6+) | ✅ | ❌ | ✅ |
| Cost Optimization | ✅ | ❌ | ❌ | ❌ |
| Compliance Built-in | ✅ | ❌ | ❌ | ✅ |
| Team Collaboration | ✅ | ❌ | ✅ | ✅ |
| Plugin Ecosystem | ✅ (21 hooks) | ❌ | Limited | ❌ |
| Self-Hosted Option | ✅ | ✅ | ❌ | ✅ |

---

## Use Cases

### 🏥 Healthcare - HIPAA Compliance Testing
- Built-in HIPAA requirement mapping
- Audit trail generation
- PII detection and validation

### 💰 FinTech - SOX Compliance Auditing
- Financial control testing
- Change management validation
- Segregation of duties verification

### 🛒 E-commerce - End-to-End Testing
- Payment gateway validation
- Multi-currency testing
- Localization verification

### 🏢 Enterprise - Migration Testing
- Data integrity validation
- Schema comparison
- Performance benchmarking

---

## Example Workflows

Representative scenarios showing how Synthesize fits real testing problems. These are illustrative, not customer results.

### Compliance-Heavy Release (Healthcare)
**Problem**: Manual HIPAA compliance testing is slow and hard to evidence every release.

**With Synthesize**:
- Express compliance tests in plain English, mapped to HIPAA requirements
- Generate audit-ready traceability reports automatically
- Self-healing keeps the suite green as the application changes

### Reducing Test Maintenance (FinTech)
**Problem**: Brittle, framework-locked tests consume a large share of QA time.

**With Synthesize**:
- Regenerate the same intent across Selenium, Playwright, or Cypress — no rewrites
- Self-healing repairs broken locators instead of failing the run
- Cost controls (caching, local models, smart routing) reduce LLM spend

### Stabilizing CI/CD (E-commerce)
**Problem**: Flaky tests intermittently block the pipeline.

**With Synthesize**:
- Flakiness detection identifies and quarantines unstable tests
- Smart orchestration runs the right tests in parallel
- Trend analytics surface regressions before they reach main

---

## Community

- 💬 **Discord**: [Join our community](https://discord.gg/synthesize)
- 🐦 **Twitter**: [@synthesize_dev](https://twitter.com/synthesize_dev)
- 📝 **Blog**: [blog.synthesize-ai.dev](https://blog.synthesize-ai.dev)
- 📺 **YouTube**: [Tutorials and demos](https://youtube.com/@synthesize_dev)
- 📖 **Docs**: [docs.synthesize-ai.dev](https://docs.synthesize-ai.dev)

---

## Contributing

We welcome contributions! See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.

**Areas we need help with:**
- 🔌 New framework plugins
- 📚 Documentation improvements
- 🐛 Bug reports and fixes
- ✨ Feature requests
- 🌍 Internationalization

---

## Roadmap

### Q1 2026
- ✅ Plugin system v2 with 21 lifecycle hooks
- ✅ Cost optimization (70% reduction)
- ✅ Multi-framework support (6 frameworks)
- ✅ Synthesize Studio web app
- ✅ Studio Dashboard with Recharts analytics
- ✅ Per-user LLM cost tracking (PostgreSQL-backed)
- ✅ Framework-aware chat (6 frameworks)
- ✅ Run history with RF report/log persistence
- ✅ Page analyzer with DOM element extraction
- ✅ 12 in-app tutorials (beginner to advanced)
- ✅ Enhanced regression detection (moving averages, confidence intervals)
- ✅ ML failure predictions (weighted prediction model)
- ✅ Live report preview tab in Studio
- ✅ OWASP ZAP security scanner integration
- ✅ WebSocket real-time run progress
- ✅ Team sharing with email invites (Amazon SES)
- ✅ GitHub export (push projects to GitHub repos)
- ✅ Stripe billing (free/pro/team plans)

### Q2 2026
- 🚧 Mobile testing (Appium plugin)
- 🚧 GitHub App for PR integration
- 🚧 Visual regression testing
- 🚧 Satellite agent for firewall-restricted environments

### Q3 2026
- 📋 AI-powered test maintenance
- 📋 Auto-documentation generation
- 📋 Multi-language support (beyond English)
- 📋 Performance testing suite

### Q4 2026
- 📋 Enterprise SSO backends (SAML/LDAP identity provider integration)
- 📋 Advanced compliance frameworks
- 📋 On-premise deployment automation
- 📋 Global plugin registry

[View Full Roadmap →](https://synthesize-ai.dev/roadmap)

---

## FAQ

**Q: Do I need to know Python/JavaScript?**
A: No. Write tests in plain English. Synthesize generates the code.

**Q: What frameworks are supported?**
A: 6 built-in (Robot, Playwright, Cypress, Selenium, Cucumber, pytest), plus community plugins.

**Q: How accurate is the AI generation?**
A: 90%+ accuracy with proper context. Self-healing handles edge cases.

**Q: Can I use my own test framework?**
A: Yes! Build a plugin in ~100 lines of Python.

**Q: Is my test data secure?**
A: Yes. All data processing is local. LLM calls are encrypted. Self-hosted option available.

**Q: What's the pricing?**
A: Open source is free. Pro is $49/user/month. Enterprise is custom pricing.

**Q: How does self-healing work?**
A: When a test fails, Synthesize analyzes the failure, attempts fixes (new locators, retry logic), and reports confidence.

**Q: Can I migrate existing tests?**
A: Yes! Import from Robot Framework, Selenium, Cucumber, or plain English.

[More FAQs →](https://synthesize-ai.dev/faq)

---

## License

MIT © 2026 Synthesize. Built by Lanier Developments.

---

## Credits

Built with:
- [Claude](https://anthropic.com) - AI interpretation
- [Robot Framework](https://robotframework.org) - Test execution
- [Playwright](https://playwright.dev) - Browser automation
- [Next.js](https://nextjs.org) - Web application
- [FastAPI](https://fastapi.tiangolo.com) - Backend API

---

**[Get Started](https://synthesize-ai.dev/docs/quickstart) • [Documentation](https://docs.synthesize-ai.dev) • [Plugins](https://plugins.synthesize-ai.dev) • [Discord](https://discord.gg/synthesize)**
