Metadata-Version: 2.4
Name: perception-mcp
Version: 0.1.3
Summary: Perception & verification MCP server for AI agents: media_assert (declarative pass/fail QC for generated media), image diff (pixelmatch + SSIM), readable text tiles for small print & handwriting, video tools (frames on demand, scenes, QC defects, pacing dynamics, attention heatmaps, scene diff), and full audio DSP (pitch, BPM, key, LUFS, spectrograms).
Project-URL: Repository, https://github.com/egorthinks/perception-mcp
Author: Egor Fedorov
License: MIT License
        
        Copyright (c) 2026 Egor Fedorov
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
License-File: LICENSE
Keywords: agents,assert,audio,dsp,handwriting,image-diff,loudness,lufs,mcp,media,ocr,perception,qc,scenes,spectrogram,ssim,verification,video,visual-regression
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Multimedia :: Graphics
Classifier: Topic :: Multimedia :: Sound/Audio :: Analysis
Classifier: Topic :: Software Development :: Quality Assurance
Requires-Python: >=3.10
Requires-Dist: mcp>=1.0
Requires-Dist: numpy>=1.24
Requires-Dist: pillow>=10.0
Provides-Extra: all
Requires-Dist: faster-whisper>=1.0; extra == 'all'
Requires-Dist: fonttools>=4.47; extra == 'all'
Requires-Dist: librosa>=0.10; extra == 'all'
Requires-Dist: matplotlib>=3.7; extra == 'all'
Requires-Dist: mutagen>=1.46; extra == 'all'
Requires-Dist: numpy<2.5; extra == 'all'
Requires-Dist: pikepdf>=8.0; extra == 'all'
Requires-Dist: pyloudnorm>=0.1; extra == 'all'
Requires-Dist: pypdfium2>=4.28; extra == 'all'
Requires-Dist: rapidocr-onnxruntime>=1.3; extra == 'all'
Requires-Dist: rtree>=1.0; extra == 'all'
Requires-Dist: scikit-image>=0.22; extra == 'all'
Requires-Dist: scipy>=1.11; extra == 'all'
Requires-Dist: soundfile>=0.12; extra == 'all'
Requires-Dist: trimesh>=4.0; extra == 'all'
Requires-Dist: yt-dlp>=2024.1.1; extra == 'all'
Provides-Extra: audio
Requires-Dist: librosa>=0.10; extra == 'audio'
Requires-Dist: matplotlib>=3.7; extra == 'audio'
Requires-Dist: mutagen>=1.46; extra == 'audio'
Requires-Dist: numpy<2.5; extra == 'audio'
Requires-Dist: pyloudnorm>=0.1; extra == 'audio'
Requires-Dist: soundfile>=0.12; extra == 'audio'
Provides-Extra: docs
Requires-Dist: pikepdf>=8.0; extra == 'docs'
Requires-Dist: pypdfium2>=4.28; extra == 'docs'
Requires-Dist: scikit-image>=0.22; extra == 'docs'
Requires-Dist: scipy>=1.11; extra == 'docs'
Provides-Extra: fonts
Requires-Dist: fonttools>=4.47; extra == 'fonts'
Provides-Extra: image
Requires-Dist: scikit-image>=0.22; extra == 'image'
Requires-Dist: scipy>=1.11; extra == 'image'
Provides-Extra: mesh
Requires-Dist: matplotlib>=3.7; extra == 'mesh'
Requires-Dist: rtree>=1.0; extra == 'mesh'
Requires-Dist: scipy>=1.11; extra == 'mesh'
Requires-Dist: trimesh>=4.0; extra == 'mesh'
Provides-Extra: ocr
Requires-Dist: rapidocr-onnxruntime>=1.3; extra == 'ocr'
Requires-Dist: scikit-image>=0.22; extra == 'ocr'
Requires-Dist: scipy>=1.11; extra == 'ocr'
Provides-Extra: transcribe
Requires-Dist: faster-whisper>=1.0; extra == 'transcribe'
Requires-Dist: librosa>=0.10; extra == 'transcribe'
Requires-Dist: matplotlib>=3.7; extra == 'transcribe'
Requires-Dist: mutagen>=1.46; extra == 'transcribe'
Requires-Dist: numpy<2.5; extra == 'transcribe'
Requires-Dist: pyloudnorm>=0.1; extra == 'transcribe'
Requires-Dist: soundfile>=0.12; extra == 'transcribe'
Provides-Extra: youtube
Requires-Dist: librosa>=0.10; extra == 'youtube'
Requires-Dist: matplotlib>=3.7; extra == 'youtube'
Requires-Dist: mutagen>=1.46; extra == 'youtube'
Requires-Dist: numpy<2.5; extra == 'youtube'
Requires-Dist: pyloudnorm>=0.1; extra == 'youtube'
Requires-Dist: soundfile>=0.12; extra == 'youtube'
Requires-Dist: yt-dlp>=2024.1.1; extra == 'youtube'
Description-Content-Type: text/markdown

# perception-mcp

<!-- mcp-name: io.github.egorthinks/perception -->

Perception & verification MCP server for AI agents — the measuring
instruments a text-and-image model lacks. Where a model can only *look*
at media, this server lets it **measure, compare and verify**: local
recognized algorithms (no cloud APIs), compact JSON answers, and PNG
renderings the model can look at. The successor and superset of
[audio-dsp-mcp](https://github.com/egorthinks/DSPmcp).

## The idea

An agent that generates or edits media works blind: it can't hear the
podcast it produced, can't tell whether two screenshots differ by a real
regression or antialiasing noise, can't read the exact color of a
button. Every tool here follows one formula:

> opaque file → compact JSON with numbers → optional PNG for the
> model's vision → an honest "not sure" when the signal isn't there.

## Tools

### media_assert — declarative QC (the headline)

One call turns the whole server into a test harness for media:

```json
media_assert("episode.mp3", expect={
  "duration_s": [1790, 1810],
  "lufs": [-17, -13],
  "no_clipping": true,
  "no_silence_longer_s": 2.0
})
```

→ `{"passed": false, "summary": "3 of 4 checks passed (1 failed)",
"checks": [{"check": "lufs", "status": "fail", "actual": -9.2, ...}]}`

Checks run only for what you ask, files are decoded once, an unknown
check name returns the full supported list (instant self-correction),
and a check that doesn't apply to the file type is reported `skipped`,
not failed. Generate → assert → regenerate.

Video files get their own keys (needs ffmpeg): `resolution`, `fps`,
`has_audio`, `no_black_longer_s`, `no_freeze_longer_s`,
`no_silence_longer_s`, `min_sharpness`, `duration_s`.

### find_media — from "the photo I just sent you" to a real path

A chat attachment never reaches an MCP server: the protocol has no
client→server file channel, so the model gets rendered pixels — for
images not even a filename. But the original file almost always
exists on the user's machine (they attached it from there).
`find_media` closes the gap: it searches the user's home (Spotlight
on macOS) and the standard folders — Downloads, Desktop, Documents,
Pictures, Movies, Music — by type, recency and name fragment,
newest first. So instead of demanding paths, the model asks
"IMG_4021.jpg, 4 min ago, 3.1 MB — this one?". Only names, sizes
and dates are returned; contents are never read. Extra folders via
the `PERCEPTION_SCAN_DIRS` env var.

### image ( `[image]` extra )

- **`compare_images`** — perceptual pixel diff (pixelmatch YIQ metric)
  + SSIM structural score, with a global-shift compensation via phase
  correlation (a 1-px scroll no longer lights up the whole diff),
  antialiasing separation, and changed-pixel clusters with exact
  bounding boxes. Returns an A | B | highlight PNG panel.
  Tolerance modes: `strict` / `normal` / `layout`.
- **`measure_image`** — exact pixel colors (hex/rgb/CSS name) at given
  points, pixel distances between points, dominant palette (k-means in
  CIELab).
- **`zoom_grid`** — a microscope: magnify a region with a labeled
  coordinate grid in absolute source coordinates, so tiny text, 1-px
  borders and exact positions become visible and referenceable.

### ocr ( `[ocr]` extra )

- **`extract_text_boxes`** — printed text with exact pixel coordinates
  per line (RapidOCR: PaddleOCR's DBNet + CRNN via ONNX, models ship
  in the wheel — no downloads, no cloud).
- **`prepare_text_tiles`** — the small-print / handwriting reader.
  A vision LLM fails on tiny text because tokenization gives each
  glyph too few pixels; this tool fixes that with classic document
  preprocessing — CLAHE contrast, projection-profile deskew, Sauvola
  ink mask, line segmentation — and returns each line as a big
  magnified labeled strip **for the calling model to read itself**.
  No recognition model in the loop, so it works for any script and
  any handwriting the calling model can read.
- **`check_contrast`** — WCAG 2.x audit of a screenshot: per text
  line, exact W3C contrast ratio + AA/AAA pass/fail.
- media_assert keys: `contains_text`, `min_contrast_ratio`.

### audio ( `[audio]` extra — the full audio-dsp-mcp toolset )

`detect_pitch`, `estimate_tempo`, `detect_key` (+ Camelot),
`analyze_loudness` (LUFS, ITU-R BS.1770), `detect_silence`,
`analyze_audio`, `compare_audio`, `render_spectrogram`,
`render_waveform`, `describe_audio`, `get_metadata`; with the
`[youtube]` extra `get_youtube_transcript`, with `[transcribe]` local
Whisper `transcribe_audio`.

### video ( no extra — just ffmpeg on PATH )

- **`describe_video`** — the whole video in one call: stream facts,
  shot structure (cuts, avg shot length), audio profile, speech
  transcript (with `[transcribe]`), and a contact sheet with a
  timestamped frame per scene.
- **`get_frames`** — SEE the footage: exact `timestamps`, an even
  sweep (`every`), or a storyboard of one moment
  (`around=[t, span, fps]`); many frames pack into one timecoded
  contact sheet.
- **`check_video`** — QC with exact timestamps: black stretches,
  frozen picture, silent audio (ffmpeg blackdetect / freezedetect /
  silencedetect), soft focus (Laplacian variance).
- **`video_dynamics`** — pacing on one timeline: cut rhythm, motion
  energy, EBU R128 momentary loudness, hook metrics for the opening
  seconds (first cut, cold-start flags) + a chart image.
- **`video_heatmap`** — WHERE things happen: motion accumulation and
  spectral-residual saliency (Hou & Zhang) drawn over a real frame,
  with a 3x3 grid breakdown for crop/caption decisions.
- **`compare_videos`** — scene-level diff of two cuts
  (Needleman-Wunsch alignment + SSIM): identical / changed / added /
  removed scenes.
- **`detect_scenes`** — shot boundaries as timestamps.

Every analysis tool takes `offset_s` / `duration_s`, so any part of a
long file is reachable; reported timestamps are absolute.

**Sources**: every tool takes a local path, an http(s) URL (25 MB
cap), or a base64 data URI. Share links resolve automatically —
Google Drive ("Anyone with the link"), Dropbox and tmpfiles.org
links are rewritten to direct downloads, and with `[youtube]`,
video-platform page URLs resolve via yt-dlp. For chat clients where
attachments never reach the server (a protocol-level limit), the
served MCP instructions teach the model an escalation ladder:
client-provided path → `find_media` on the user's disk → ask for a
path (with the copy-path gesture) → ask for a share link. All
failures come back as an `"error"` field — the server never crashes
the conversation.

### docs ( `[docs]` extra )

- **`compare_pdf`** — page-level visual diff of two PDFs (pypdfium2
  render → the same perceptual diff as screenshots): layout shifts,
  swapped images, font substitution; diff panels for changed pages.
- **`check_pdf`** — will it open everywhere: structural validation
  (pikepdf/qpdf), encryption, non-embedded fonts, pages that crash
  rendering, blank pages.

### fonts ( `[fonts]` extra )

- **`render_font_specimen`** — SEE a typeface and verify it: rendered
  specimen sheet (pangrams for every script the font covers) + exact
  cmap coverage (`"cyrillic": "complete" / "partial (74%)" / "none"`),
  metrics, variable axes.

### mesh ( `[mesh]` extra )

- **`render_model`** — seven orthographic views of a 3D model on one
  sheet (GLB/GLTF/STL/OBJ/PLY), pure-CPU rendering, works headless.
- **`check_mesh`** — printability QC: watertightness, winding,
  degenerate faces, volume, extents, `printable` verdict with reasons.
- **`compare_mesh`** — geometric diff: ICP alignment + two-sided
  point-to-surface distances (chamfer / Hausdorff) + overlay render.

## Install

Core is lean; add what you need: `[image]`, `[ocr]`, `[audio]`,
`[youtube]`, `[transcribe]`, `[docs]`, `[fonts]`, `[mesh]`, or
`[all]`.

### Claude Code

```bash
claude mcp add perception -- uvx --from "perception-mcp[all]" perception-mcp
```

### Claude Desktop / any MCP client

```json
{
  "mcpServers": {
    "perception": {
      "command": "uvx",
      "args": ["--from", "perception-mcp[all]", "perception-mcp"]
    }
  }
}
```

## Roadmap

MCP sampling mode, A/V sync detection, mesh wall-thickness analysis.

## License

MIT
