Metadata-Version: 2.4
Name: videre-mcp
Version: 0.2.0
Summary: MCP server that bridges vision models to text-only coding models using Florence-2
Author: videre-mcp contributors
License-Expression: MIT
Keywords: florence-2,image-description,mcp,ocr,vision
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.11
Requires-Dist: cairosvg>=2.7.0
Requires-Dist: einops>=0.7.0
Requires-Dist: mcp>=1.0.0
Requires-Dist: mss>=9.0.0
Requires-Dist: pillow>=10.0.0
Requires-Dist: timm>=0.9.0
Requires-Dist: torch>=2.0.0
Requires-Dist: transformers==4.48.3
Provides-Extra: deep
Requires-Dist: accelerate>=0.30.0; extra == 'deep'
Requires-Dist: bitsandbytes>=0.43.0; extra == 'deep'
Provides-Extra: dev
Requires-Dist: pytest-asyncio>=0.24.0; extra == 'dev'
Requires-Dist: pytest>=8.0.0; extra == 'dev'
Provides-Extra: docling
Requires-Dist: docling>=2.0.0; extra == 'docling'
Provides-Extra: optimize
Requires-Dist: dspy-ai<3.0,>=2.5.0; extra == 'optimize'
Provides-Extra: paddle
Requires-Dist: paddleocr>=2.8.0; extra == 'paddle'
Requires-Dist: paddlepaddle>=3.0.0; extra == 'paddle'
Description-Content-Type: text/markdown

# videre-mcp

MCP server that bridges vision models to text-only coding models using Florence-2.

Non-vision LLMs can't see images — videre-mcp fixes that. It loads a Florence-2 vision model locally and exposes four MCP tools that convert images (including SVGs) and screenshots into structured text descriptions that any text-based model can consume.

```
Screenshot tool → videre-mcp (Florence-2) → Text description → Coding model
```

## Installation

```bash
pip install videre-mcp
```

Or with uv:

```bash
uv pip install videre-mcp
```

Requires Python 3.11+ and ~300MB disk space for the Florence-2-base model weights (downloaded automatically on first use).

## Usage

Add to your OpenCode configuration:

```json
{
  "mcpServers": {
    "videre-mcp": {
      "command": "videre-mcp"
    }
  }
}
```

Or run directly:

```bash
videre-mcp
# or
python -m videre_mcp
```

## Tools

### `describe_image`

Generate a natural language description of an image.

**Parameters:**
- `image_path` (str) — Path to the image file (supports PNG, JPEG, SVG)
- `detail_level` (str, optional) — `"normal"` (default) for brief caption, `"high"` for detailed description

**Example:**

```python
result = describe_image("/path/to/photo.png", detail_level="high")
# Returns:
# {
#   "description": "A sunlit meadow with wildflowers in bloom...",
#   "model": "Florence-2-base",
#   "prompt_used": "<MORE_DETAILED_CAPTION>"
# }
```

### `ocr_image`

Extract text from an image using optical character recognition.

**Parameters:**
- `image_path` (str) — Path to the image file (supports PNG, JPEG, SVG)
- `detail_level` (str, optional) — `"normal"` (default) for plain text, `"high"` for text with bounding regions

**Example:**

```python
result = ocr_image("/path/to/document.png", detail_level="high")
# Returns:
# {
#   "text": "Invoice Number 12345",
#   "regions": [
#     {"label": "Invoice Number 12345", "bbox": [10, 20, 30, 40, 50, 60, 70, 80]}
#   ]
# }
```

### `describe_screenshot`

Describe UI regions in a screenshot — designed for coding agents that need to understand screen layouts.

**Parameters:**
- `image_path` (str) — Path to the screenshot file (supports PNG, JPEG, SVG)
- `detail_level` (str, optional) — `"normal"` (default) for dense region captions, `"high"` for per-region descriptions

**Example:**

```python
result = describe_screenshot("/path/to/screenshot.png")
# Returns:
# {
#   "regions": [
#     {"bbox": [10, 20, 30, 40], "label": "search bar"},
#     {"bbox": [100, 200, 300, 250], "label": "submit button"}
#   ],
#   "model": "Florence-2-base"
# }
```

### `take_screenshot`

Capture a screenshot and optionally describe it using Florence-2. Supports multi-monitor setups via the `monitor` parameter.

**Parameters:**
- `output_path` (str, optional) — Path to save the screenshot PNG. If `None`, saves to a temp file.
- `monitor` (int, optional) — Monitor index: `0` = all monitors combined, `1` = primary, etc. (default: `0`)
- `describe` (bool, optional) — If `True`, also run `describe_screenshot` on the captured image (default: `True`)

**Example:**

```python
result = take_screenshot(monitor=1, describe=True)
# Returns:
# {
#   "path": "/tmp/tmpxxxxxx.png",
#   "width": 1920,
#   "height": 1080,
#   "monitor": 1,
#   "regions": [
#     {"label": "search bar", "bbox": [10, 20, 30, 40]},
#     ...
#   ]
# }
```

## Supported Formats

| Format | Support |
|--------|---------|
| PNG | ✅ Native |
| JPEG | ✅ Native |
| SVG | ✅ Via cairosvg rasterization |

## Requirements

- Python 3.11+
- ~300MB disk for model weights (auto-downloaded on first inference)
- Works on CPU; GPU (CUDA) is auto-detected and used if available

## Continuous Integration

The Florence-2 slow tests (real model load + inference) run on a nightly
schedule via GitHub Actions. See `.github/workflows/slow-tests.yml`.

## License

MIT — see [LICENSE](LICENSE).

### Third-party licenses

This package vendors a patched copy of Microsoft's Florence-2 processor
(`src/videre_mcp/_vendor/processing_florence2.py`) under Microsoft's MIT license.
See [src/videre_mcp/_vendor/LICENSE-Microsoft-Florence-2](src/videre_mcp/_vendor/LICENSE-Microsoft-Florence-2).
