Metadata-Version: 2.5
Name: hikari-agent
Version: 1.0.0
Summary: Hikari - Autonomous AI Desktop Agent for the Windows Operating System.
Project-URL: homepage, https://github.com/Codewithevilxd/hikari
Project-URL: repository, https://github.com/Codewithevilxd/hikari
Author-email: "Nishant Gaurav (Codewithevilxd)" <codewithevilxd@gmail.com>
License-Expression: MIT
Keywords: ai-agent,computer-use-agent,desktop-automation,gui-automation,hikari,llm,windows-os-agent
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: Microsoft :: Windows
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Software Development :: User Interfaces
Requires-Python: >=3.13
Requires-Dist: fuzzywuzzy>=0.18.0
Requires-Dist: humancursor>=1.1.5
Requires-Dist: ipykernel>=6.29.5
Requires-Dist: langchain-community>=0.3.25
Requires-Dist: langchain-google-genai>=2.1.5
Requires-Dist: langchain-groq>=0.3.4
Requires-Dist: langchain-ollama>=0.3.3
Requires-Dist: langchain-openai>=0.3.27
Requires-Dist: langchain>=0.3.25
Requires-Dist: langgraph>=0.6.4
Requires-Dist: markdownify>=1.1.0
Requires-Dist: pillow>=11.2.1
Requires-Dist: psutil>=7.0.0
Requires-Dist: pyautogui>=0.9.54
Requires-Dist: pydantic>=2.11.7
Requires-Dist: python-levenshtein>=0.27.1
Requires-Dist: rich>=14.0.0
Requires-Dist: tabulate>=0.9.0
Requires-Dist: termcolor>=3.1.0
Requires-Dist: twine>=6.1.0
Requires-Dist: uiautomation>=2.0.28
Provides-Extra: dev
Requires-Dist: pytest-mock>=3.14.0; extra == 'dev'
Requires-Dist: pytest>=8.4.1; extra == 'dev'
Requires-Dist: ruff>=0.12.1; extra == 'dev'
Description-Content-Type: text/markdown

# 🌟 Hikari (光) — Autonomous Windows AI Agent

<p align="center">
  <img src="https://img.shields.io/badge/Hikari-Autonomous%20Desktop%20Agent-8A2BE2?style=for-the-badge&logo=windows&logoColor=white" alt="Hikari">
  <img src="https://img.shields.io/badge/Author-Nishant%20Gaurav%20(Codewithevilxd)-FF4500?style=for-the-badge&logo=github&logoColor=white" alt="Author">
  <img src="https://img.shields.io/badge/Python-3.13+-3776AB?style=for-the-badge&logo=python&logoColor=white" alt="Python">
  <img src="https://img.shields.io/badge/License-MIT-green?style=for-the-badge" alt="License">
  <img src="https://img.shields.io/badge/Version-1.0.0-blue?style=for-the-badge" alt="Version">
</p>

---

## ⚡ Overview

**Hikari (光)** is a next-generation **Autonomous Computer-Use AI Agent** engineered specifically for the **Microsoft Windows** operating system. 

Unlike traditional browser-restricted agents or costly vision-only agents that burn thousands of tokens capturing full-screen screenshots on every turn, **Hikari** reads and decomposes the **Native Windows UI Accessibility Tree** directly via OS APIs. This enables lightning-fast, ultra-precise, token-efficient desktop control across any Windows application — from native Win32 apps and UWP modern apps to web browsers, terminal consoles, and complex creative suites.

Hikari thinks like an expert human operator: observing open windows, analyzing interactive elements, planning next actions, moving the cursor via natural Bezier curves, typing text, executing PowerShell scripts, and self-correcting when errors occur.

---

## 🚀 Key Features & Superpowers

- ⚡ **Native Windows Accessibility Tree Engine**: Directly extracts interactive, informative, and scrollable UI controls (buttons, inputs, menus, panes) with exact bounding boxes and coordinates without requiring heavy image processing.
- 🧠 **Chain-of-Thought State Machine**: Uses structured XML reasoning tags (`<evaluate>`, `<memory>`, `<plan>`, `<thought>`, `<action_name>`, `<action_input>`) to ensure rock-solid decision-making and loop recovery.
- 🎯 **Human-like Precision Control**: Simulates natural human mouse trajectories using Bezier curves (`SystemCursor`) to avoid robotic click detection and maintain smooth UI interaction.
- 🌐 **Multi-Model LLM Agnostic**: Fully supports Groq Cloud (ultra-low latency), Google Gemini 2.0 Flash, OpenAI GPT-4o, GPT-4o-mini, and Local Offline models via Ollama.
- 💻 **Hybrid GUI + Shell Automation**: Combines graphical mouse/keyboard interactions with direct PowerShell and CMD command execution for instant speedups.
- 🪟 **Smart Window Manager**: Detects active windows, monitors application launch statuses, switches foreground windows, and resizes layouts dynamically.
- 📋 **System Clipboard & Web Scraping**: Built-in clipboard read/write capabilities and automated webpage scraping with markdown conversion.

---

## 🏗️ System Architecture

```
                                    +-----------------------+
                                    |     User Request      |
                                    +-----------+-----------+
                                                |
                                                v
                                    +-----------------------+
                                    |   Hikari Entrypoint   |
                                    |  (CLI / Python API)   |
                                    +-----------+-----------+
                                                |
                                                v
                               +---------------------------------+
                               |   hikari.desktop.service        |
                               |  - Window Enumeration (GetApps) |
                               |  - Active Window Detection      |
                               |  - Screenshot (Optional Vision) |
                               +----------------+----------------+
                                                |
                                                v
                               +---------------------------------+
                               |     hikari.tree.service         |
                               |  - Windows Accessibility Tree   |
                               |  - Interactive Nodes (Clickable)|
                               |  - Informative Nodes (Text)     |
                               |  - Scrollable Panes (Offsets)   |
                               +----------------+----------------+
                                                |
                                                v
                               +---------------------------------+
                               |     hikari.agent.prompt         |
                               |  - Dynamic XML Context Builder  |
                               |  - Desktop State Injection      |
                               +----------------+----------------+
                                                |
                                                v
                               +---------------------------------+
                               |      LLM Reasoning Engine       |
                               |  (Groq / Gemini / GPT-4o / etc) |
                               |  Output: <thought> & <action>   |
                               +----------------+----------------+
                                                |
                                                v
                               +---------------------------------+
                               |    hikari.agent.tools.service   |
                               |  - Click / Type / Shell / Drag  |
                               |  - Launch / Switch / Shortcut   |
                               +----------------+----------------+
                                                |
                                                v
                                    +-----------------------+
                                    | Windows OS Execution  |
                                    +-----------------------+
```

---

## 🧰 Built-in Tool Catalog

Hikari features an extensible, type-safe tool registry powered by Pydantic models:

| Tool Name | Arguments / Parameters | Description |
|---|---|---|
| **`Launch Tool`** | `name: str` | Launches any application found in the Start Menu or PATH (e.g., `notepad`, `chrome`, `spotify`, `calc`). |
| **`Click Tool`** | `loc: (x, y)`, `button: left\|right\|middle`, `clicks: 0\|1\|2\|3` | Simulates human mouse clicks with hover (0), single (1), double (2), or triple (3) clicks at exact coordinates. |
| **`Type Tool`** | `loc: (x, y)`, `text: str`, `clear: true\|false`, `caret_position: start\|idle\|end`, `press_enter: true\|false` | Clicks target element, handles caret positioning, optionally clears existing text, types naturally, and sends Enter. |
| **`Shell Tool`** | `command: str` | Executes native PowerShell commands and returns terminal output, errors, and exit codes. |
| **`Switch Tool`** | `name: str` | Brings target running application window directly to the foreground. |
| **`Resize Tool`** | `name: str`, `loc: (x, y)`, `size: (w, h)` | Repositions and resizes any application window. |
| **`Scroll Tool`** | `loc: (x, y)`, `type: horizontal\|vertical`, `direction: up\|down\|left\|right`, `wheel_times: int` | Performs vertical or horizontal scroll wheel steps on active or targeted controls. |
| **`Drag Tool`** | `from_loc: (x1, y1)`, `to_loc: (x2, y2)` | Drags UI elements or files across coordinates with mouse down, trajectory move, and mouse up. |
| **`Move Tool`** | `to_loc: (x, y)` | Moves the mouse pointer smoothly without clicking. |
| **`Shortcut Tool`**| `shortcut: list[str]` | Presses keyboard shortcuts (e.g., `['ctrl', 'c']`, `['alt', 'f4']`, `['win', 'r']`). |
| **`Key Tool`** | `key: str` | Presses single special keys (e.g., `enter`, `escape`, `tab`, `backspace`, `f5`). |
| **`Clipboard Tool`**| `mode: copy\|paste`, `text: Optional[str]` | Reads or writes text to the Windows system clipboard. |
| **`Scrape Tool`** | `url: str` | Fetches webpage HTML and converts it into clean, token-efficient Markdown text. |
| **`Wait Tool`** | `duration: int` | Pauses execution for a specified duration in seconds to allow UI loading. |
| **`Done Tool`** | `answer: str` | Signals task completion and returns the final markdown summary to the user. |

---

## 💻 Installation & Setup

### Prerequisites
- **Operating System**: Windows 10 or Windows 11 (64-bit)
- **Python Version**: Python 3.13 or higher
- **Package Manager**: `uv` (recommended) or `pip`

### 1. Clone the Repository
```bash
git clone https://github.com/Codewithevilxd/hikari.git
cd hikari
```

### 2. Create Virtual Environment & Install Dependencies
Using `uv` (ultra-fast):
```bash
uv venv
uv pip install -e ".[dev]"
```

Or using standard `pip`:
```bash
python -m venv .venv
.\.venv\Scripts\activate
pip install -e ".[dev]"
```

### 3. Configure Environment Variables
Create a `.env` file in the root directory (or copy `.env-example`):

```env
# Groq Cloud API Key (Ultra-fast LLMs)
GROQ_API_KEY="your-groq-api-key-here"

# Google Gemini API Key (Recommended free tier)
GOOGLE_API_KEY="your-gemini-api-key-here"

# OpenAI API Key (Optional)
OPENAI_API_KEY="your-openai-api-key-here"
```

---

## 🎮 Usage Guide

### 1. Interactive CLI Interface
Launch the interactive terminal:
```bash
.\.venv\Scripts\python main.py
```

You will be greeted with the Hikari interactive menu:
```text
  _    _ _____ _  __     _____  _____ 
 | |  | |_   _| |/ / /\ |  __ \|_   _|
 | |__| | | | | ' / /  \| |__) | | |  
 |  __  | | | |  < / /\\ \  _  /  | |  
 | |  | |_| |_| . \ / ____ \ | \ \ _| |_ 
 |_|  |_|_____|_|\_/_/    \_\|  \_\_____|
           光 - Autonomous Windows AI Agent
     Developed by Nishant Gaurav (Codewithevilxd)

Choose your AI Model Engine:
  1. Google Gemini 2.0 Flash (Recommended - Superfast & Free Tier)
  2. OpenAI GPT-4o (High Precision Desktop Control)
  3. OpenAI GPT-4o-mini (Budget Fast Option)
  4. Groq Cloud AI (Ultra-low Latency - GPT-OSS / Qwen / Compound)
  5. Local Ollama (Private / Offline)
```

Type any task in plain English or Hindi:
- *"Open Notepad and write a 4-line poem about rain, then save it to Desktop"*
- *"Launch Chrome, search for latest space discoveries, and copy the first headline"*
- *"Open Spotify and play my favourite playlist"*
- *"Check available disk space in PowerShell and report back"*

---

### 2. Programmatic Python SDK Integration

You can easily integrate Hikari into your own Python applications and automated pipelines:

```python
import os
from dotenv import load_dotenv
from langchain_groq import ChatGroq
from hikari.agent import Agent

load_dotenv()

# 1. Initialize your preferred LLM
llm = ChatGroq(
    model_name="openai/gpt-oss-120b",
    temperature=0.1
)

# 2. Instantiate Hikari Agent
agent = Agent(
    llm=llm,
    browser="chrome",       # Preferred browser
    use_vision=False,       # Set True for vision-augmented workflows
    auto_minimize=True,     # Auto minimize console during execution
    max_steps=15,           # Max autonomous decision cycles
    consecutive_failures=3  # Retry limit for stuck states
)

# 3. Execute any desktop task
task = "Open Calculator and compute 1024 * 768"
result = agent.invoke(task)

print("Task Result:", result.content)
```

---

## 🧪 Running Automated Tests

Hikari comes with a comprehensive test suite containing **159 unit tests** covering all modules, registry services, desktop bindings, and tree parsing:

```bash
.\.venv\Scripts\python -m pytest tests/ -v
```

Expected Output:
```text
============================= 159 passed in 1.94s =============================
```

---

## 📂 Project Directory Structure

```text
hikari/
├── hikari/
│   ├── __init__.py                  # Package root exporter
│   ├── agent/
│   │   ├── __init__.py
│   │   ├── service.py               # Main Agent orchestrator & LangGraph state machine
│   │   ├── state.py                 # Pydantic agent memory states
│   │   ├── utils.py                 # Regex XML parser & image serialisation
│   │   ├── views.py                 # Core Agent data schemas
│   │   ├── prompt/
│   │   │   ├── __init__.py
│   │   │   ├── service.py           # Dynamic prompt formatter
│   │   │   ├── system.md            # Hikari Core System Persona
│   │   │   ├── action.md            # Action decision prompt
│   │   │   ├── answer.md            # Final answer prompt
│   │   │   └── observation.md       # Desktop observation prompt
│   │   ├── registry/
│   │   │   ├── __init__.py
│   │   │   ├── service.py           # Dynamic tool execution registry
│   │   │   └── views.py             # ToolResult & metadata models
│   │   └── tools/
│   │       ├── __init__.py
│   │       ├── service.py           # Native tool functions (Click, Type, Shell, etc.)
│   │       └── views.py             # Pydantic tool argument schemas
│   ├── desktop/
│   │   ├── __init__.py
│   │   ├── config.py                # Avoided & excluded system applications
│   │   ├── service.py               # Windows process & window management
│   │   └── views.py                 # App & DesktopState view models
│   └── tree/
│       ├── __init__.py
│       ├── config.py                # Accessibility control type maps
│       ├── service.py               # Native Accessibility Tree parser
│       ├── utils.py                 # Bounding box & coordinate helpers
│       └── views.py                 # Interactive & scrollable node schemas
├── tests/
│   ├── conftest.py                  # Pytest fixtures & mock configuration
│   └── unit/                        # 14 unit test suites (159 tests)
├── .env-example                     # Template for API keys
├── LICENSE                          # MIT License
├── main.py                          # Interactive Rich CLI application
├── main_optimized.py                # Optimized fast runner
└── pyproject.toml                   # Project metadata & package specification
```

---

## 📜 License

This project is licensed under the **MIT License** — see the [LICENSE](LICENSE) file for details.

---

## 👤 Author & Maintainer

**Nishant Gaurav (Codewithevilxd)**
- 🐙 **GitHub**: [@Codewithevilxd](https://github.com/Codewithevilxd)
- 📧 **Email**: [codewithevilxd@gmail.com](mailto:codewithevilxd@gmail.com)
- 🌐 **Project Repository**: [https://github.com/Codewithevilxd/hikari](https://github.com/Codewithevilxd/hikari)

---

<p align="center">
  <b>Hikari (光)</b> — <i>Illuminating Windows Desktop Automation.</i>
</p>
