Metadata-Version: 2.5
Name: omniscout
Version: 0.4.6.3
Summary: OmniScout CLI: local-first multi-browser automation, semantic search, and research for AI agents
Project-URL: Homepage, https://omniscout.xyz
Project-URL: Repository, https://github.com/sriramramnath/omniscout
Project-URL: Documentation, https://docs.omniscout.xyz
Project-URL: Changelog, https://github.com/sriramramnath/omniscout/blob/main/cli/CHANGELOG.md
Project-URL: Issues, https://github.com/sriramramnath/omniscout/issues
Author: OmniScout
License: Modified MIT License
        
        Copyright (c) 2026 OmniScout
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
        
        Our only modification part is that, if the Software (or any derivative works
        thereof) is used for any of your products or services, you shall prominently
        display "Powered by OmniScout" on the user interface of such product or
        service.
License-File: LICENSE
Keywords: agent,cli,patchright,research,scraping,search
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.11
Requires-Dist: aiohttp>=3.9
Requires-Dist: babel>=2.18.0
Requires-Dist: certifi>=2026.6.17
Requires-Dist: flask-babel>=4.0.0
Requires-Dist: flask>=3.1.3
Requires-Dist: httpx-socks[asyncio]>=0.10.0
Requires-Dist: httpx[http2]>=0.27
Requires-Dist: isodate>=0.7.2
Requires-Dist: jinja2>=3.1.6
Requires-Dist: lxml>=6.1.1
Requires-Dist: markdown-it-py>=4.2.0
Requires-Dist: markdownify>=0.13
Requires-Dist: msgspec>=0.21.1
Requires-Dist: nltk>=3.8
Requires-Dist: patchright>=1.61.2
Requires-Dist: platformdirs>=4.2
Requires-Dist: pydantic>=2.7
Requires-Dist: pygments>=2.20.0
Requires-Dist: python-dateutil>=2.9.0.post0
Requires-Dist: pyyaml>=6.0.3
Requires-Dist: qdrant-client>=1.9
Requires-Dist: rich>=13.7
Requires-Dist: selectolax>=0.3.21
Requires-Dist: sentence-transformers>=2.7
Requires-Dist: sniffio>=1.3.1
Requires-Dist: sumy>=0.11
Requires-Dist: tomli>=2.0; python_version < '3.11'
Requires-Dist: torch>=2.2
Requires-Dist: trafilatura>=1.12
Requires-Dist: transformers>=4.40
Requires-Dist: typer>=0.12
Requires-Dist: typing-extensions>=4.16.0
Requires-Dist: valkey>=6.1.1
Requires-Dist: whitenoise>=6.12.0
Provides-Extra: dev
Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
Requires-Dist: pytest-xdist>=3.5; extra == 'dev'
Requires-Dist: pytest>=8; extra == 'dev'
Requires-Dist: respx>=0.21; extra == 'dev'
Requires-Dist: ruff>=0.5; extra == 'dev'
Provides-Extra: train
Requires-Dist: accelerate>=0.30; extra == 'train'
Requires-Dist: datasets>=2.19; extra == 'train'
Requires-Dist: peft>=0.11; extra == 'train'
Requires-Dist: trl>=0.9; extra == 'train'
Description-Content-Type: text/markdown

<div align="center">

# OmniScout

Current release: `0.4.6.3`

**Local-first browser automation, semantic search & research for AI agents.**

No cloud APIs. No hosted sessions. No lock-in. Just a fast, local HTTP daemon in your terminal.



![Python 3.11+](https://shields.io/badge/python-%3E=3.11-4584d3?style=flat-square&logo=python&logoColor=white)
[![License](https://img.shields.io/badge/license-Modified%20MIT-393939?style=flat-square)](LICENSE)
![PyPI - Version](https://img.shields.io/pypi/v/omniscout?style=flat-square&logo=pypi&logoColor=white&color=blue) ![PyPI Downloads](https://img.shields.io/pepy/dt/omniscout?style=flat-square&logo=pypi&logoColor=white&color=green)
<br>
[![macOS](https://img.shields.io/badge/macOS-8E8E93?style=flat-square&logo=apple&logoColor=white)]()
[![Linux](https://img.shields.io/badge/Linux-FFD700?style=flat-square&logo=linux&logoColor=black)]()
<br>
[![Website](https://img.shields.io/badge/Website-omniscout.xyz-2b7a78?style=flat-square)](https://omniscout.xyz) [![Docs](https://img.shields.io/badge/Docs-docs.omniscout.xyz-2b7a78?style=flat-square)](https://docs.omniscout.xyz)

![demo-terminal.png](https://pxdrop.online/raw/d9b4dbuhv1ts73baueag)


</div>

```bash
pip install omniscout
```

Requirements: Python 3.11 or newer and a Chromium-based browser such as Chrome,
Brave, Edge, Vivaldi, Opera, Arc, or Chromium. OmniScout stores browser profiles,
indexes, caches, and downloaded models locally. The first search or answer may
download and warm an embedding model.

Verify the installation and inspect available commands:

```bash
omniscout --version
omniscout --help
omniscout computer doctor       # diagnose desktop automation (macOS)
```

## Quick Start

Warm the local models, search, open a result in the managed browser, and inspect
the page for accessibility references:

```bash
omniscout warmup
omniscout answer "latest AI news"
omniscout open
omniscout snapshot --refs-only
omniscout browser click '@e3'
```

For a direct URL, use the browser command group:

```bash
omniscout daemon start
omniscout browser navigate https://news.ycombinator.com
omniscout browser snapshot --refs-only
omniscout browser click '@e3'
omniscout browser close
```

Configure default browser:

```bash
omniscout settings set browser brave
```

Install agent skill files so Claude Code, Cursor, and Codex auto-discover OmniScout:

```bash
omniscout install --skill
```

The installer currently writes skill files for Claude Code, Cursor, Codex CLI,
and Gemini CLI. Other agents can use the bundled `SKILL.md` manually.

For deterministic agent integration, use `--json` or set `OMNISCOUT_JSON=1`:

```bash
omniscout --json answer "What is local-first software?"
export OMNISCOUT_JSON=1
```

When you need to discover a command or option, use the built-in help:

```bash
omniscout --help
omniscout computer --help
omniscout computer ui --help
```

---

## Documentation Overview

The full documentation lives at [docs.omniscout.xyz](https://docs.omniscout.xyz) and is organized as follows:

### CLI

| Page | What it covers |
|------|----------------|
| [Overview](https://docs.omniscout.xyz/cli/overview/) | What OmniScout is, what you get (daemon, backends, `@eN` refs, search/extract/research/graph engines, Probe Zero), design principles, installation, hello world, data layout, and configuration |
| [Agents](https://docs.omniscout.xyz/cli/agents/) | Drop-in prompts and skill files for Claude Code, Cursor, Codex, and Kimi; multi-step agent loops; JSON contract; trace / replay / watch; error kinds; skill template |
| [Examples](https://docs.omniscout.xyz/cli/examples/) | Copy-paste recipes: knowledge graphs, research, form fill, login + profile reuse, CAPTCHA, network capture, multi-step workflows, screenshots, PDFs, sessions, tabs, search → extract → answer, and Python/shell integration |
| [Commands](https://docs.omniscout.xyz/cli/commands/) | Full CLI reference — every command, flag, and JSON shape: daemon lifecycle, browser action vocabulary, tab/network/console management, search/extract/research/graph options, profiles, settings, workflow commands, response envelope, and environment variables |
| [Architecture](https://docs.omniscout.xyz/cli/architecture/) | Daemon, backends (Patchright vs Extension), `@eN` snapshot refs, action queue, snapshot generations, action history & replay, session restore, event stream, and all core layers (commands, engines, daemon, storage, config, models) with data-flow diagrams |
| [Roadmap](https://docs.omniscout.xyz/cli/roadmap/) | Planned features — multi-provider search, smart extraction, page summaries, and MCP server |
| [Troubleshooting](https://docs.omniscout.xyz/cli/troubleshooting/) | Every common failure mode and fix: `@eN` refs, login, CAPTCHA, extension backend, installation, runtime, search, extraction, research, browser automation, profiles, daemon, performance, cache, JSON output, and configuration issues |
| [Probe Zero](https://docs.omniscout.xyz/cli/probe-zero/) | OmniScout's local answer engine — what it does, how to enable it, Classic vs Probe Zero, and gold-benchmark metrics |

### SDK

| Page | What it covers |
|------|----------------|
| [SDK Overview](https://docs.omniscout.xyz/sdk/) | What the SDK is, key features, installation, quick example, and common use cases |
| [Python API Reference](https://docs.omniscout.xyz/sdk/api/) | Complete Python API docs — Search, Extraction, Crawler, Research, Browser, Daemon Client, Configuration, Models, Logging, async support, testing, performance tips, and troubleshooting |

---

## What It Is

OmniScout is a self-contained **terminal-native interface for the web**. Think of it as a local browser brain for AI agents — enabling them to search, browse, extract structured data, and remember everything without ever leaving your laptop.

A long-lived daemon at `127.0.0.1:7720` provides sub-second browser actions, a warm embedding model, and a local vector store. It communicates over HTTP/JSON and works with your existing Chromium browser (Chrome, Brave, Edge, Vivaldi, etc.). Local state and processing stay on your machine, while search, crawling, extraction, and browser automation still access the external websites or providers you request.

---

## Why OmniScout?

| Feature | Status |
| :--- | :--- |
| **Local First** | No hosted browser sessions; state and processing are local by default |
| **Semantic Search** | Searches the web and re-ranks results using local sentence-transformers embeddings |
| **Long-lived Daemon** | Sub-second per-action latency with persistent sessions |
| **Real Browser** | Drives your real Chrome/Brave; keeps cookies, logins, and extensions |
| **Semantic Memory** | Remember and search your browsing history with vector embeddings |
| **Rich Extraction** | Pull structured data from any URL with zero LLM calls |
| **Agent-Ready** | Every command speaks JSON — hook it up to any AI framework |

---

## Core Commands

The following is the complete beginner-oriented command map. Run any command with
`--help` for its flags and subcommands.

| Command | Purpose |
| --- | --- |
| `search` | Search the web, optionally with local semantic reranking. |
| `answer` | Retrieve web results and synthesize an answer; preferred when you need an answer rather than a URL list. |
| `map` | Crawl a site and produce a URL map. |
| `warmup` | Preload embedding and answer models in the daemon. |
| `remember`, `memory` | Store and semantically search remembered visits and notes. |
| `extract` | Return readable or structured data from a URL or query. |
| `open`, `snapshot`, `context`, `reset` | Open pages, inspect `@eN` refs, view workflow state, and reset continuity. |
| `auto` | Route a one-line request to the best command. |
| `research`, `graph` | Run full research pipelines or build entity knowledge graphs. |
| `settings`, `install` | Configure OmniScout and install agent skills or browser support. |
| `browser` | Direct Chromium automation for complex web workflows. |
| `computer` | Native desktop automation for apps, files, clipboard, and windows. |
| `daemon`, `session`, `profile` | Manage the service, persistent sessions, and browser profiles. |
| `index`, `extract-jobs`, `crawl`, `monitor` | Manage indexes, background jobs, crawls, and URL change monitoring. |
| `record`, `macro`, `replay`, `workflow` | Record, reuse, replay, and export workflows. |
| `benchmark` | Benchmark answer modes. |

### Search

![Search](https://pxdrop.online/raw/d9bpb86hv1ts73bauf3g)

### 1. Search & Research

```bash
# Semantic web search
omniscout search "state of local AI agents 2026"

# Summarized one-sentence answer (no LLM)
omniscout answer "Who is the president?" --depth balanced

# Full research pipeline (search → crawl → extract → rerank → summarize)
omniscout research "emerging quantum computing startups 2026"

# Open the local OmniScout Search UI (bundled SearXNG at 127.0.0.1:8888)
omniscout search ui
```

### Browser

![Browser](https://pxdrop.online/raw/d9bpbnuhv1ts73bauf4g)

### 2. Browser Automation

```bash
# Navigate and identify elements with accessibility tree refs
omniscout browser navigate https://news.ycombinator.com
omniscout browser snapshot --refs-only
omniscout browser click '@e3'

# Screenshots (full page, delayed, or centered)
omniscout browser screenshot --full-length --out state.png
omniscout browser screenshot --delay 2 --out after-load.png

# Capture network and console logs
omniscout browser network list
omniscout browser console tail
```

### Computer

![Computer](https://pxdrop.online/raw/d9bpbumhv1ts73bauf50)

### 3. Desktop Automation

```bash
# macOS MVP: open apps/files/URLs and type into the focused window
omniscout computer navigate /Applications/Notes.app
omniscout computer wait 300
omniscout computer type "Meeting notes"
omniscout computer key cmd+s

# Clipboard, screenshots, and window management
omniscout computer clipboard set "Hello from Scout"
omniscout computer screenshot --out /tmp/desktop.png
omniscout computer window list
```

Desktop automation currently supports macOS via native `open`, AppleScript,
`screencapture`, and clipboard tools. Commands use the same daemon JSON envelope
and `--session` model as browser automation.

### Extract

![Extract](https://pxdrop.online/raw/d9bpbhehv1ts73bauf40)

### 4. Content Extraction

```bash
# Clean Markdown or structured JSON from any page
omniscout extract https://example.com
omniscout extract -q "SpaceX founder" --format structured --fields founder

# Exa-style schema-driven extraction (no LLM)
omniscout extract https://stripe.com/pricing \
  --schema-inline '{"type":"object","properties":{"pricing":{"type":"string"}}}'
```

### 5. Knowledge Graphs

```bash
# Map a company or person into a structured Unicode tree
omniscout graph "Cursor"              # search web and extract
omniscout graph "Cursor" -w cursor.com --data  # crawl site directly
omniscout graph "Cursor" --llm        # optional LLM overlay on evidence
```

### 6. Browser Memory

```bash
# Remember and semantically search your browsing history
omniscout remember https://example.com/blog/post
omniscout memory "neural networks in production"
```

### Desktop automation reference

`computer` supports `navigate`, `type`, `key`, `screenshot`, `wait`, `doctor`,
`clipboard`, `window`, `ui`, and `backend`. The `computer ui` group supports
`snapshot`, `click`, `drag`, and `is`; use its `@eN` references for element-level
actions. Desktop automation currently supports macOS through native OS tools.

### Reusable workflow example

```bash
omniscout record start --name my_macro
omniscout browser navigate https://example.com
omniscout browser snapshot --refs-only
omniscout record stop my_macro
omniscout macro run my_macro
```

---

## Development

```bash
cd cli
python -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"
pytest
ruff check .
```

Tests live in `cli/tests/`. Integration tests are marked with `integration` and
may start subprocesses or bind local ports. Keep browser profiles and generated
model/cache data outside the repository.

## Architecture

```mermaid
flowchart LR
    A["AI agent (Claude / Cursor / Codex / Kimi)"] --> B["OmniScout CLI<br/>(typer + rich)"]
    B -->|"HTTP POST"| C["Daemon<br/>(127.0.0.1:7720)"]
    C --> D["Patchright Backend"]
    C --> E["Extension Backend<br/>(opt-in)"]
    C --> F["Embed Service<br/>(warm model)"]
    B --> G["Search / Extract / Research / Graph"]
    G --> H["Local Qdrant"]
    G --> I["SearXNG / Crawler"]
    D --> J["Chrome / Brave / Edge / Vivaldi"]
```

For contributors:

```
cli/omniscout/
  app.py              # Typer root
  commands/           # CLI subcommands
  daemon/             # HTTP server, backends, replay
  engines/            # browser, search, research, extractor, crawler
  store/              # SQLite cache, sessions, workflow, memory
  models.py           # Pydantic JSON contracts
```

---

## JSON & Agent Integration

Every command speaks JSON. Set `OMNISCOUT_JSON=1` and `stdout` becomes a structured payload.

```bash
export OMNISCOUT_JSON=1
omniscout search "robotics simulators"
```

Or talk to the daemon directly:

```bash
curl -X POST http://127.0.0.1:7720/command \
  -H 'Content-Type: application/json' \
  -d '{"action":"navigate","args":{"url":"https://example.com"},"session":"demo"}'
```

---

## Configuration

Create a `config.toml` in your config directory (e.g. `~/.config/omniscout/config.toml` on Linux, `~/Library/Application Support/omniscout/config.toml` on macOS):

```toml
default_source = "searxng"   # searxng | ddg | wikipedia | web
search_limit = 10
research_results = 8
request_throttle_seconds = 1.0
embedding_model = "sentence-transformers/all-MiniLM-L6-v2"
browser = "chrome"                    # chrome | edge | brave | vivaldi | opera | arc | chromium | custom
# browser_executable = "/path/to/browser"  # only needed for 'custom'
```

Or configure via CLI:

```bash
omniscout settings browsers
omniscout settings set browser brave
omniscout settings show
```

### Environment Variables

| Variable | Purpose |
| --- | --- |
| `OMNISCOUT_JSON=1` | Force JSON output on every command |
| `OMNISCOUT_DAEMON_AUTO_START=0` | Don't auto-start the daemon |
| `OMNISCOUT_DAEMON_PORT` | Daemon port (default: 7720) |
| `OMNISCOUT_DATA_DIR` | Override the default data directory |
| `OMNISCOUT_BROWSER` | Browser ID (overrides config) |
| `OMNISCOUT_EMBED_LOCAL_ONLY=0` | Allow runtime Hugging Face fetches |
| `TWOCAPTCHA_API_KEY` | CAPTCHA solver API key |

Legacy `HARNESS_*` equivalents accepted.

---

## On-disk State

| Path | Purpose |
| --- | --- |
| `profiles/` | Persistent Chrome user-data-dirs |
| `qdrant/` | Embedded vector index |
| `models/sentence-transformers/` | Prefetched embedding model |
| `memory.sqlite` | Browser memory (visits + notes) |
| `sessions.sqlite` | Long-lived browser session registry |
| `cache/pages/` | Content-hashed HTML cache |
| `daemon/` | PID, port, logs, action history, session restore |

---

## Supported By

OmniScout works with any AI agent that can run shell commands. Skill files are auto-installed with `omniscout install --skill` wherever a well-known directory exists.

**Agents with verified skill directories:**
- **Claude Code** — `~/.claude/skills/`
- **OpenAI Codex CLI** — `~/.codex/skills/`
- **Cursor** — `~/.cursor/skills-cursor/`
- **Gemini CLI / Antigravity** — `~/.gemini/config/skills/`
- **Pi** — paste `SKILL.md` into the system prompt
- **OpenCode** — paste `SKILL.md` into the system prompt
- **Windsurf** — paste `SKILL.md` into the system prompt
- **Cline** — paste `SKILL.md` into the system prompt
- **Roo Code** — paste `SKILL.md` into the system prompt
- **Amp** — paste `SKILL.md` into the system prompt
- **Replit Agent** — paste `SKILL.md` into the system prompt
- **AiderDesk** — paste `SKILL.md` into the system prompt
- **AstrBot** — paste `SKILL.md` into the system prompt
- **Droid (Factory)** — paste `SKILL.md` into the system prompt
- **Goose** — paste `SKILL.md` into the system prompt
- **Factory** — paste `SKILL.md` into the system prompt

**Also works with:** GitHub Copilot (Agent Mode), any custom agent with shell access, or any coding assistant that can invoke the `omniscout` / `scout` CLI binary.

The CLI is the interface. If your agent runs `bash`, it runs OmniScout.

---

## License

Modified MIT — see [LICENSE](LICENSE). Products built on OmniScout must prominently display **Powered by OmniScout** on the user interface.
