Metadata-Version: 2.4
Name: qontinui
Version: 0.1.1
Summary: Model-based GUI automation with AI-enhanced perception
License: MIT
License-File: LICENSE
Keywords: gui,automation,ai,testing,rpa
Author: Joshua Spinak
Author-email: jspinak@alum.mit.edu
Requires-Python: >=3.12,<4.0
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Software Development :: Testing
Provides-Extra: all
Provides-Extra: sam
Requires-Dist: accelerate (>=0.25,<0.26)
Requires-Dist: croniter (>=5.0,<6.0)
Requires-Dist: easyocr (>=1.7,<2.0)
Requires-Dist: faiss-cpu (>=1.7,<2.0)
Requires-Dist: lark (>=1.1,<2.0)
Requires-Dist: mss (>=9.0,<10.0)
Requires-Dist: networkx (>=3.2,<4.0)
Requires-Dist: numpy (>=1.26,<2.0)
Requires-Dist: opencv-python-headless (>=4.9,<5.0)
Requires-Dist: pillow (>=10.2,<11.0)
Requires-Dist: prometheus-client (>=0.19,<0.20)
Requires-Dist: psutil (>=5.9,<6.0)
Requires-Dist: pydantic (>=2.11.9,<3.0.0)
Requires-Dist: pydantic-settings (>=2.0,<3.0)
Requires-Dist: pynput (>=1.7,<2.0)
Requires-Dist: python-dotenv (>=1.0,<2.0)
Requires-Dist: scikit-learn (>=1.3,<2.0)
Requires-Dist: segment-anything (>=1.0,<2.0) ; extra == "sam" or extra == "all"
Requires-Dist: sqlalchemy (>=2.0,<3.0)
Requires-Dist: structlog (>=24.0,<25.0)
Requires-Dist: supervision (>=0.26,<0.27)
Requires-Dist: timm (>=0.9,<0.10)
Requires-Dist: tokenizers (>=0.15,<0.16)
Requires-Dist: torch (>=2.2,<3.0)
Requires-Dist: torchvision (>=0.17)
Requires-Dist: transformers (>=4.36,<5.0)
Requires-Dist: transitions (>=0.9,<0.10)
Requires-Dist: ultralytics (>=8.0,<9.0)
Project-URL: Documentation, https://qontinui.github.io
Project-URL: Homepage, https://github.com/qontinui/qontinui
Project-URL: Repository, https://github.com/qontinui/qontinui
Description-Content-Type: text/markdown

# Qontinui

[![PyPI version](https://badge.fury.io/py/qontinui.svg)](https://badge.fury.io/py/qontinui)
[![Python 3.12+](https://img.shields.io/badge/python-3.12+-blue.svg)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)

A Python library for model-based GUI automation with intelligent state management and visual recognition.

## Research Foundation

Based on [**Model-based GUI Automation**](https://link.springer.com/article/10.1007/s10270-025-01319-9) published in Springer's Software and Systems Modeling journal (October 2025).

The research provides:
- Mathematical proof of complexity reduction (exponential → polynomial)
- First testable approach to GUI automation (unit tests, integration tests)
- Formal framework for robust visual APIs for RL agents
- Enables reliable dataset generation for AI training

## Overview

Qontinui enables building robust GUI automation through:
- **Model-based state management** using [MultiState](https://github.com/qontinui/multistate) | [Docs](https://qontinui.github.io/multistate/)
- **Visual recognition** with OpenCV template matching
- **JSON configuration** for defining automation workflows
- **Cross-platform support** (Windows, macOS, Linux)

Qontinui is a Python port of [Brobot](https://github.com/jspinak/brobot), a Java library for GUI automation (2018-2024).

## Installation

### From PyPI (Recommended)

```bash
pip install qontinui
```

### From Source (Development)

```bash
# Clone the repository
git clone https://github.com/jspinak/qontinui.git
cd qontinui

# Install with Poetry
poetry install

# Or with pip
pip install -e .
```

### Dependencies

Qontinui requires:
- **[MultiState](https://github.com/jspinak/multistate)** - Multi-state state management
- **OpenCV** - Image template matching
- **PyAutoGUI/pynput** - Input control

## Quick Start

### JSON Configuration

Create an automation configuration in JSON:

```json
{
  "version": "1.0",
  "states": [
    {
      "name": "LoginScreen",
      "stateImages": [
        {
          "imageId": "login_button",
          "threshold": 0.9
        }
      ]
    }
  ],
  "processes": [
    {
      "name": "Login",
      "actions": [
        {
          "type": "CLICK",
          "target": {"type": "image", "imageId": "login_button"}
        },
        {
          "type": "TYPE",
          "text": "username@example.com"
        }
      ]
    }
  ]
}
```

### Python API

```python
from qontinui.json_executor import JSONRunner

# Initialize runner
runner = JSONRunner()

# Load configuration
runner.load_configuration("automation_config.json")

# Execute automation
success = runner.run(monitor_index=0)
```

### Desktop Application

Use [Qontinui Runner](https://github.com/qontinui/qontinui-runner) for a GUI interface to create and run automations.

## Architecture

```
qontinui/
├── src/qontinui/
│   ├── json_executor/       # JSON configuration execution
│   ├── model/               # State, Transition, Image models
│   ├── hal/                 # Hardware Abstraction Layer
│   └── multistate_adapter/  # MultiState integration
```

## Features

### Current
- ✅ JSON-based automation configuration
- ✅ Template-based image matching
- ✅ Multi-state state management
- ✅ Process and state machine execution modes
- ✅ Cross-platform input control (PyAutoGUI, pynput)
- ✅ Hardware abstraction layer for multiple backends

### Planned
- 🔄 AI-enhanced visual recognition (SAM, CLIP)
- 🔄 Domain-specific language (DSL)
- 🔄 Advanced Brobot migration tools
- 🔄 Cloud execution via qontinui-web

## Testing

Run the test suite:

```bash
# Run all tests
poetry run pytest

# Run with coverage
poetry run pytest --cov=src/qontinui

# Run specific test file
poetry run pytest tests/json_executor/test_json_runner.py
```

## Contributing

We welcome contributions! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.

Please note that this project is released with a [Code of Conduct](CODE_OF_CONDUCT.md). By participating in this project you agree to abide by its terms.

Qontinui is a faithful port of Brobot. When contributing, please preserve Brobot's architecture and behavior.

## Documentation

- **GitHub**: [github.com/qontinui/qontinui](https://github.com/qontinui/qontinui)
- **Issues**: [GitHub Issues](https://github.com/qontinui/qontinui/issues)
- **MultiState Docs**: [qontinui.github.io/multistate](https://qontinui.github.io/multistate/)
- **Research Paper**: [Springer SoSyM](https://link.springer.com/article/10.1007/s10270-025-01319-9)

## License

MIT License - See [LICENSE](LICENSE) file for details.

## Acknowledgments

- [Brobot](https://github.com/jspinak/brobot) - The original Java library
- [MultiState](https://github.com/jspinak/multistate) - Multi-state state management
- [PyAutoGUI](https://github.com/asweigart/pyautogui) - GUI automation
- [OpenCV](https://opencv.org/) - Computer vision

## Related Projects

- **[qontinui-runner](https://github.com/qontinui/qontinui-runner)** - Desktop application (Rust/TypeScript)
- **[qontinui-api](https://github.com/qontinui/qontinui-api)** - REST API bridge (enables custom frontends)
- **[qontinui-web](https://qontinui.com)** - Web-based visual builder (launching Feb 2026)
- **[multistate](https://github.com/qontinui/multistate)** - State management library | [Docs](https://qontinui.github.io/multistate/)
- **[Brobot](https://github.com/jspinak/brobot)** - Original Java implementation (2018-2024)

