Metadata-Version: 2.4
Name: layoutlens
Version: 1.0.0
Summary: AI-powered UI testing framework with natural language visual validation
Home-page: https://github.com/layoutlens/layoutlens
Author: LayoutLens Team
Author-email: Gaurav Sood <contact@gsood.com>
Maintainer-email: Gaurav Sood <contact@gsood.com>
License: MIT
Project-URL: Homepage, https://github.com/matmulai/layoutlens
Project-URL: Bug Reports, https://github.com/matmulai/layoutlens/issues
Project-URL: Documentation, https://layoutlens.readthedocs.io/
Project-URL: Source, https://github.com/matmulai/layoutlens
Project-URL: Changelog, https://github.com/matmulai/layoutlens/blob/main/CHANGELOG.md
Keywords: ui testing,visual regression,ai,computer vision,web testing,layout testing
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Testing
Classifier: Topic :: Internet :: WWW/HTTP :: Dynamic Content
Classifier: Topic :: Multimedia :: Graphics :: Capture :: Screen Capture
Classifier: Operating System :: OS Independent
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: openai>=1.0.0
Requires-Dist: playwright>=1.40.0
Requires-Dist: beautifulsoup4>=4.12.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: Pillow>=10.0.0
Provides-Extra: dev
Requires-Dist: pytest>=7.0.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
Requires-Dist: pytest-mock>=3.10.0; extra == "dev"
Requires-Dist: pytest-xdist>=3.0.0; extra == "dev"
Requires-Dist: black>=23.0.0; extra == "dev"
Requires-Dist: flake8>=6.0.0; extra == "dev"
Requires-Dist: mypy>=1.0.0; extra == "dev"
Requires-Dist: isort>=5.12.0; extra == "dev"
Provides-Extra: docs
Requires-Dist: sphinx>=5.0.0; extra == "docs"
Requires-Dist: sphinx-rtd-theme>=1.0.0; extra == "docs"
Provides-Extra: test
Requires-Dist: pytest>=7.0.0; extra == "test"
Requires-Dist: pytest-cov>=4.0.0; extra == "test"
Requires-Dist: pytest-mock>=3.10.0; extra == "test"
Requires-Dist: pytest-xdist>=3.0.0; extra == "test"
Requires-Dist: pytest-asyncio>=0.21.0; extra == "test"
Requires-Dist: coverage[toml]>=7.0.0; extra == "test"
Dynamic: author
Dynamic: home-page
Dynamic: license-file
Dynamic: requires-python

# LayoutLens: AI-Enabled UI Test System

[![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Build Status](https://github.com/matmulai/layoutlens/workflows/tests/badge.svg)](https://github.com/matmulai/layoutlens/actions)

Write visual UI tests using natural language to validate web layouts, accessibility compliance, and user interface consistency across devices. LayoutLens combines computer vision AI with automated screenshot testing to provide comprehensive UI validation.

## 🎯 Key Features

- **Natural Language Testing**: Write UI tests in plain English
- **Multi-Viewport Testing**: Automatically test responsive designs across devices
- **Accessibility Validation**: Built-in WCAG compliance checking
- **Screenshot Comparison**: Visual regression testing with AI-powered analysis
- **Form Validation Testing**: Comprehensive form interaction and validation testing
- **CI/CD Integration**: Easy integration with existing development workflows

## 📊 Benchmark Results

Our comprehensive benchmark suite demonstrates LayoutLens effectiveness across diverse UI patterns:

### Test Suite Coverage

| Benchmark File | Description | Size | Elements Tested |
|----------------|-------------|------|----------------|
| **ecommerce_product.html** | E-commerce product page | 13.6KB | Product galleries, pricing, responsive layout |
| **dashboard.html** | Analytics dashboard | 16.7KB | Data grids, charts, complex layouts |
| **contact_form.html** | Complex form validation | 20.5KB | Form elements, validation, accessibility |
| **accessibility_showcase.html** | WCAG compliance demo | 31.9KB | Screen readers, keyboard nav, contrast |
| **css_grid_showcase.html** | Advanced CSS layouts | 27.6KB | CSS Grid, Flexbox, responsive breakpoints |
| **mobile_first_responsive.html** | Mobile-first design | 32.1KB | Progressive enhancement, touch interactions |

### Performance Results

**Query Generation Performance:**
- ✅ **100%** success rate across all benchmark files (4/4 files processed)
- ✅ **246 total queries** generated across benchmark suite
- ✅ **61.5 queries average** per file (range: 33-94 queries)
- ✅ **<100ms** average generation time per page

**AI Analysis Accuracy:**
- ✅ **100%** OpenAI integration success rate (8/8 tests passed)
- ✅ **Real-time analysis** of layout, accessibility, and semantic markup
- ✅ **Intelligent responses** with specific reasoning for each query
- ✅ **Context-aware** analysis using HTML structure and content

**Content Structure Recognition:**
```
📊 Verified Element Detection Results:
File                     | Size   | Forms | Images | Links | Headings | Inputs | Buttons | Queries
ecommerce_product.html   | 13.3KB |   0   |   5    |   5   |    3     |   1    |    2    |   34
contact_form.html        | 20.0KB |   1   |   0    |   2   |    1     |   13   |    1    |   33
accessibility_showcase   | 31.1KB |   1   |   1    |   7   |   10     |   10   |    5    |   85
css_grid_showcase.html   | 26.9KB |   0   |   0    |   31  |   16     |   0    |    2    |   94
```

**Sample AI Analysis Results:**
- **Semantic Markup**: "No. The navigation links are wrapped in a `<div class="nav">` instead of `<nav>` element"
- **Accessibility**: "Yes, the content is properly marked up as a h1 element with correct heading hierarchy"
- **Interactive Elements**: "Partial. HTML includes interactive elements with CSS focus styles defined"

### Real-World Test Scenarios

**✅ E-commerce Testing**
- Product image galleries and thumbnails
- Pricing displays and discount calculations
- Mobile-responsive product layouts
- Add-to-cart functionality validation

**✅ Dashboard Analytics**
- Complex data table structures
- Chart and graph layout validation
- Multi-column responsive grids
- Interactive dashboard components

**✅ Form Validation**
- Progressive form enhancement
- Real-time validation feedback
- Accessibility compliance (WCAG 2.1 AA)
- Mobile-friendly form interactions

**✅ Responsive Design**
- Mobile-first progressive enhancement
- Breakpoint testing across 6+ screen sizes
- Touch target size validation
- Viewport meta tag optimization

### Sample Test Queries Generated

```yaml
Accessibility Tests:
  - "Are all form elements properly labeled and accessible?"
  - "Is the color contrast sufficient for readability?"
  - "Do all images have appropriate alt text?"

Layout Tests:  
  - "Is the page layout responsive across different screen sizes?"
  - "Are interactive elements easily clickable on mobile devices?"
  - "Is the heading hierarchy logical and well-structured?"

Visual Tests:
  - "Does the navigation menu collapse properly on mobile?"
  - "Are the product images displayed in the correct aspect ratio?"
  - "Is the form validation feedback clearly visible?"
```

## 🚀 Quick Start

### Installation

```bash
pip install layoutlens
playwright install chromium  # Required for screenshots
```

### Basic Usage

```python
from layoutlens import LayoutLens

# Initialize the testing framework
tester = LayoutLens()

# Test a single page with auto-generated queries
result = tester.test_page(
    "homepage.html",
    viewports=["mobile_portrait", "desktop"],
    auto_generate_queries=True
)

print(f"Success rate: {result.success_rate:.2%}")
print(f"Tests passed: {result.passed_tests}/{result.total_tests}")
```

### CLI Usage

```bash
# Test with automatic query generation
layoutlens test homepage.html --viewports mobile,desktop

# Test with custom queries
layoutlens test homepage.html --query "Is the navigation menu properly aligned?"

# Run full test suite
layoutlens suite tests/ui_tests.yaml
```

### Advanced Features

```python
# Compare two page versions
comparison = tester.compare_pages(
    "before_redesign.html",
    "after_redesign.html",
    query="Are the layouts visually consistent?"
)

# Create and run test suites
suite = tester.create_test_suite(
    name="Homepage Tests",
    description="Comprehensive homepage validation",
    test_cases=[
        {
            "name": "Mobile Homepage",
            "html_path": "homepage.html",
            "queries": ["Is the menu collapsed on mobile?"],
            "viewports": ["mobile_portrait"]
        }
    ]
)

results = tester.run_test_suite(suite)
```

## 🧪 Running Benchmarks

Test LayoutLens with our comprehensive benchmark suite:

```bash
# Clone the repository
git clone https://github.com/matmulai/layoutlens.git
cd layoutlens

# Set up environment
export OPENAI_API_KEY="your-key-here"
pip install -e .

# Run individual benchmarks
layoutlens test benchmarks/ecommerce_product.html
layoutlens test benchmarks/accessibility_showcase.html --viewports mobile,tablet,desktop

# Generate comprehensive benchmark report
python scripts/benchmark/run_full_evaluation.py
```

## 📋 Framework Architecture

The repository includes both legacy components and the modern LayoutLens framework:

**Modern Framework (`layoutlens/`):**
- `core.py`: Enhanced LayoutLens class with user-friendly API
- `config.py`: Comprehensive configuration management
- `cli.py`: Command-line interface for easy integration

**Testing Infrastructure (`scripts/`):**
- `testing/page_tester.py`: Main testing orchestrator
- `testing/screenshot_manager.py`: Multi-viewport screenshot capture
- `testing/query_generator.py`: Intelligent test query generation
- `benchmark/benchmark_generator.py`: Automated benchmark data creation

**Benchmark Suite (`benchmarks/`):**
- 6 comprehensive HTML test pages covering real-world scenarios
- CSV datasets for batch testing and comparison
- README with detailed testing guidelines

## 🔧 Configuration

LayoutLens supports flexible configuration via YAML files or environment variables:

```yaml
# layoutlens_config.yaml
llm:
  model: "gpt-4o-mini"
  api_key: "${OPENAI_API_KEY}"

viewports:
  mobile_portrait:
    width: 375
    height: 667
    device_scale_factor: 2
    is_mobile: true
  
  desktop:
    width: 1920
    height: 1080
    device_scale_factor: 1
    is_mobile: false

testing:
  parallel_execution: true
  auto_generate_queries: true
  screenshot_format: "png"
```

## 🤝 Contributing

We welcome contributions! Please see our [Contributing Guide](CONTRIBUTING.md) for details.

### Development Setup

```bash
# Clone and set up development environment
git clone https://github.com/matmulai/layoutlens.git
cd layoutlens
python -m venv venv
source venv/bin/activate
pip install -e .
pip install -r requirements-dev.txt

# Run tests
make test

# Run linting
make lint

# Run full development checks
make full-check
```

## 📄 License

LayoutLens is released under the [MIT License](LICENSE).

## 🙏 Acknowledgments

- Built with [Playwright](https://playwright.dev/) for reliable browser automation
- Powered by [OpenAI GPT-4 Vision](https://openai.com/research/gpt-4v-system-card) for intelligent layout analysis
- Uses [BeautifulSoup](https://www.crummy.com/software/BeautifulSoup/) for HTML parsing and analysis

## 📧 Support

- 📖 [Documentation](https://layoutlens.readthedocs.io/)
- 🐛 [Bug Reports](https://github.com/matmulai/layoutlens/issues)
- 💬 [Discussions](https://github.com/matmulai/layoutlens/discussions)
- 🔗 [Homepage](https://github.com/matmulai/layoutlens)

---

*LayoutLens: Making UI testing as simple as describing what you see.* ✨
