Metadata-Version: 2.4
Name: sonicmatch-mcp
Version: 0.2.1
Summary: Pick BGM that doesn't fight the voiceover. Watches the video, returns a hook + ffmpeg ducking spec, prints the license.
Author: js713-lab
License-Expression: MIT
Project-URL: Homepage, https://github.com/js713-lab/sonic-match-mcp
Project-URL: Repository, https://github.com/js713-lab/sonic-match-mcp
Project-URL: Issues, https://github.com/js713-lab/sonic-match-mcp/issues
Project-URL: Changelog, https://github.com/js713-lab/sonic-match-mcp/blob/main/CHANGELOG.md
Project-URL: Documentation, https://github.com/js713-lab/sonic-match-mcp#readme
Keywords: mcp,music,bgm,video,ffmpeg,instagram,reels,royalty-free
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Multimedia :: Sound/Audio
Classifier: Topic :: Multimedia :: Video
Classifier: Typing :: Typed
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: mcp>=1.9
Requires-Dist: pydantic<3,>=2.6
Requires-Dist: pydantic-settings<3,>=2.2
Requires-Dist: httpx<1,>=0.27
Requires-Dist: python-dotenv<2,>=1.0
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: ruff>=0.8; extra == "dev"
Requires-Dist: tomli>=2.0; python_version < "3.11" and extra == "dev"
Provides-Extra: gemini
Requires-Dist: google-genai>=1.0; extra == "gemini"
Provides-Extra: local-vl
Requires-Dist: faster-whisper>=1.0; extra == "local-vl"
Requires-Dist: scenedetect>=0.6; extra == "local-vl"
Requires-Dist: librosa>=0.10; extra == "local-vl"
Requires-Dist: numpy>=1.26; extra == "local-vl"
Provides-Extra: embeddings
Requires-Dist: lancedb>=0.13; extra == "embeddings"
Provides-Extra: all
Requires-Dist: sonicmatch-mcp[embeddings,gemini,local-vl]; extra == "all"
Dynamic: license-file

# Sonicmatch

<!-- mcp-name: io.github.js713-lab/sonicmatch-mcp -->

[![CI](https://github.com/js713-lab/sonic-match-mcp/actions/workflows/ci.yml/badge.svg)](https://github.com/js713-lab/sonic-match-mcp/actions/workflows/ci.yml)
[![PyPI](https://img.shields.io/pypi/v/sonicmatch-mcp)](https://pypi.org/project/sonicmatch-mcp/)
[![License: MIT](https://img.shields.io/github/license/js713-lab/sonic-match-mcp)](LICENSE)
[![MCP Registry](https://img.shields.io/badge/MCP_Registry-listed-6b4fbb)](https://registry.modelcontextprotocol.io/v0.1/servers/io.github.js713-lab%2Fsonicmatch-mcp/versions/latest)
[![Python 3.10+](https://img.shields.io/badge/python-3.10%2B-blue.svg)](https://www.python.org/downloads/)

**Video-native MCP server for license-safe BGM.** Watches the footage — not the script — and returns a shortlist, a 12–20s hook, and an ffmpeg ducking spec.

<p align="center">
  <img src="https://github.com/js713-lab/sonic-match-mcp/raw/main/docs/banner.jpg" alt="Sonicmatch — video in, license-safe BGM out. Ingest, Analyze, Match, Mix." width="100%">
</p>

Package / CLI: [`sonicmatch-mcp`](https://github.com/js713-lab/sonic-match-mcp). A [Model Context Protocol](https://modelcontextprotocol.io) server for Claude Desktop, Cursor, Codex, and other MCP clients. Drop an Instagram Reel, YouTube Short, or TikTok-style clip. Get royalty-free / Creative Commons matches with the license printed on every row.

Video-to-BGM already exists. The wedge is not “I also match music”:

- it watches the **footage**, not the script
- it returns a **hook window** + ffmpeg ducking spec
- it is **agent-native**
- it **prints the license** instead of lying

Catalog quality will kill or save this. More tools will not.

```
Video or URL in
  → scene / mood / pace / speech analysis
  → license-safe BGM shortlist
  + beat/cut hints
  + optional mix preview
```

Do **not** treat this as “script in → YouTube Music search out.” That already exists (`mcp-bgm-recommender`). Sonicmatch watches the **video**.

| You own | You do not own |
|---|---|
| Local file / public URL ingest | Platform music licenses |
| Mood, energy curve, speech vs silence, scene cuts | Meta/TikTok “trending audio” graph |
| CC / royalty-free catalogs + optional paid adapters | Spotify / IG official libraries |
| Ranked tracks, preview URLs, mix spec, ffmpeg | Auto-publish to Instagram |

**North star:** `ingest_video` → `analyze_video_music` → `recommend_bgm` → `preview_mix` → `export_mix_spec`

## License warning (read this)

- The **code** is MIT.
- **Every track has its own license.** It is printed on every recommendation.
- Nothing here is an official Instagram sticker, TikTok Commercial Music Library track, or YouTube Audio Library API result.
- Do not recommend commercial pop unless the adapter is explicitly a **user-owned licensed library**.
- CC-BY still needs attribution. CC-BY-NC is **not** ok for ads / shops. Non-commercial tracks are **never auto-recommended**.
- For ads / shops, wire a **user-owned** Artlist / Epidemic JSON (`examples/user_library.example.json`). Do not scrape those sites.
- Content ID can still hit you if you point at the wrong source. A CC label is not a waiver.

## Demo

> I dropped `./clip.mp4`. Analyze it for an Instagram Reel and recommend 5 instrumental BGMs. Then mix the top pick with ducking and give me the ffmpeg command.

That prompt is the product. Install below, wire Claude Desktop, Cursor, or Codex, and paste it.

## Quick start

Requires **Python 3.10+** and `ffmpeg` / `ffprobe` on PATH. `yt-dlp` is optional and **off by default** (`SONICMATCH_ALLOW_YTDLP=0`) because platform extractors break and may violate ToS. Prefer a local file.

The PyPI project is `sonicmatch-mcp`. The console script is `sonicmatch-mcp`, which is what `uvx` and `pipx` run. `python -m sonicmatch` is the same server. The PyPI badge stays empty until the first tag is published. Install from git until then. Maintainer steps: [PUBLISH.md](PUBLISH.md).

### Works today (git)

```bash
uvx --from git+https://github.com/js713-lab/sonic-match-mcp.git sonicmatch-mcp
```

```bash
pip install git+https://github.com/js713-lab/sonic-match-mcp.git
sonicmatch-mcp --version   # or: python -m sonicmatch --version
```

### After the first PyPI release

```bash
uvx sonicmatch-mcp
# or
pipx install sonicmatch-mcp
sonicmatch-mcp --version
```

### From a clone (editable + tests)

```bash
git clone https://github.com/js713-lab/sonic-match-mcp.git
cd sonic-match-mcp
python3 -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"
cp .env.example .env   # optional keys

# stdio (Claude Desktop / Cursor / Codex)
sonicmatch-mcp

# streamable HTTP (web editors). No authentication. Keep it on loopback.
sonicmatch-mcp --http --port 8765
```

```bash
# same clone, uv
uv venv && uv pip install -e ".[dev]"
uv run sonicmatch-mcp
```

```bash
pytest          # generates tiny color mp4s with ffmpeg
ruff check src tests
```

The seed catalog is about 20 Reel-editor rows (cafe, product, talking-head, travel, food, fashion, event) plus a vocal fixture and a CC-BY-NC fixture. It runs without API keys. Gemini, Jamendo, and Freesound are optional and degrade with a note in the tool response. Seed rows have no hosted audio on purpose — `preview_mix` synthesizes a demo bed. For real ads, point `SONICMATCH_LIBRARY_PATH` / `EPIDEMIC_LIBRARY_PATH` / `ARTLIST_LIBRARY_PATH` at JSON you already licensed.

| Variable | Required | What it does |
|---|---|---|
| `GEMINI_API_KEY` | no | Gemini video understanding for `analyze_video_music` |
| `JAMENDO_CLIENT_ID` | no | Jamendo search |
| `FREESOUND_API_KEY` | no | Freesound beds and loops |
| `SONICMATCH_LIBRARY_PATH` | no | User-owned library JSON. Also `EPIDEMIC_LIBRARY_PATH`, `ARTLIST_LIBRARY_PATH` |
| `SONICMATCH_ALLOW_YTDLP` | no | `1` turns on platform URL ingest. Default `0` |
| `SONICMATCH_MAX_DOWNLOAD_MB` | no | Download cap. Default 200 |
| `SONICMATCH_HTTP_HOST` / `SONICMATCH_HTTP_PORT` | no | `--http` bind. Default `127.0.0.1:8765` |

## Features

- **Watches the picture.** Gemini video understanding when `GEMINI_API_KEY` is set; otherwise local ffmpeg / audio heuristics (optional Whisper, PySceneDetect, librosa).
- **License on every row.** CC / royalty-free / user-owned library. Non-commercial tracks are never auto-recommended. Generated beds stay out of auto recs.
- **Editor-shaped output.** 12–20s hook in/out, speech ducking, ffmpeg recipe, mix spec for CapCut / Premiere / DaVinci / your agent.
- **Agent-native MCP.** stdio for Claude Desktop and Cursor; streamable HTTP for web editors. Never ships raw multi-MB video through the payload — you get an `asset_id`.
- **Works without API keys.** 20-track seed catalog for Reels. Optional Jamendo, Freesound, and user-owned Artlist / Epidemic JSON.
- **Cuts on the beat.** `suggest_cuts` snaps scene cuts to a BPM grid and returns EDL-ish intro / peak / outro.
- **Brand lock.** Save BPM / mood / no-vocals kits and pass `brand_kit=…` into `recommend_bgm`.
- **Series, not one-offs.** `analyze_batch` (max 20) clusters mood and returns one shared mini-playlist.
- **Honest about closed graphs.** No fake “trending audio,” official IG stickers, or Content-ID-safe stamps. yt-dlp platform ingest is opt-in.

## Tools

| Tool | What it does |
|---|---|
| `status` | ffmpeg / keys / seed count / day-1 risk gates |
| `ingest_video` | Local path or HTTPS URL → `asset_id` (never video bytes). Platform URLs need `SONICMATCH_ALLOW_YTDLP=1` |
| `analyze_video_music` | Mood, energy curve, speech, scenes, hook window, BPM, search queries |
| `recommend_bgm` | 3–7 ranked tracks + why + license + hook in/out |
| `search_music` | Free-text / BPM / mood over seed + optional catalogs |
| `get_track` | One track’s metadata, license, attribution, URLs |
| `preview_mix` | Hook trim, loop, optional ducking → preview files + ffmpeg + mix spec |
| `export_mix_spec` | Mix spec + ffmpeg + attribution (no render unless `render=true`) |
| `suggest_cuts` | Beat grid, snapped scene cuts, EDL, intro / peak / outro |
| `generate_bed` | Demo bed marked `source=generated`. Requires `i_understand_not_commercially_cleared=true` |
| `save_brand_kit` | Persist BPM / moods / no-vocals for `recommend_bgm(brand_kit=…)` |
| `analyze_batch` | Up to 20 clips → mood cluster + shared mini-playlist |

Also ships a prompt template: **“Score this video like an IG music sticker.”**

### Product rules (Instagram-like, not Instagram)

- Prefer **instrumental** when `speech_coverage > 0.25`
- Recommend a **hook window**, not the whole song
- Show **why** (`cuts at 0.8s average, 112 BPM, warm gold hour`)
- Always return **license + attribution text**
- 3–7 tracks, not 40
- User can override mood / genre / no-lyrics / platform / energy
- Never claim “cleared for Instagram official sticker” unless it actually is

## Example

Prompt:

> I dropped `./clip.mp4`. Analyze it for an Instagram Reel and recommend 5 instrumental BGMs. Then mix the top pick with ducking and give me the ffmpeg command.

The agent should call, in order:

1. `ingest_video` with the local path. The result is an `asset_id`, not video bytes.
2. `analyze_video_music` with `platform_hint` set to `instagram_reel`.
3. `recommend_bgm` with that `asset_id` and `max_results` of 5.
4. `preview_mix` on the top track with ducking, then `export_mix_spec`.

A recommendation row carries `score`, `reason`, `suggested_in_out` (the hook), `ducking`, `license_ok_for_platform`, and `content_id_risk` (`unknown` or `likely`, never `cleared`). Seed fixtures include `seed_golden_hour_latte` (CC0-1.0) and `seed_indie_porch_110` (CC-BY-4.0). They have no hosted audio. `preview_mix` synthesizes a demo bed. CC-BY-NC rows are searchable and are not auto-recommended.

## Claude Desktop

`claude_desktop_config.json`. From git, no clone:

```json
{
  "mcpServers": {
    "sonicmatch": {
      "command": "uvx",
      "args": [
        "--from",
        "git+https://github.com/js713-lab/sonic-match-mcp.git",
        "sonicmatch-mcp"
      ],
      "env": {
        "GEMINI_API_KEY": "",
        "JAMENDO_CLIENT_ID": "",
        "FREESOUND_API_KEY": ""
      }
    }
  }
}
```

After a clone, `command` can be the absolute path of `.venv/bin/sonicmatch-mcp`. After the PyPI release, `args` is just `["sonicmatch-mcp"]`. Copy-paste files: [`examples/claude_desktop.mcp.json`](examples/claude_desktop.mcp.json), [`examples/claude_desktop.pypi.mcp.json`](examples/claude_desktop.pypi.mcp.json).

## Cursor

`.cursor/mcp.json` (project) or `~/.cursor/mcp.json`. From git, no clone — [`examples/cursor.mcp.json`](examples/cursor.mcp.json):

```json
{
  "mcpServers": {
    "sonicmatch": {
      "command": "uvx",
      "args": [
        "--from",
        "git+https://github.com/js713-lab/sonic-match-mcp.git",
        "sonicmatch-mcp"
      ],
      "env": {
        "GEMINI_API_KEY": "",
        "JAMENDO_CLIENT_ID": "",
        "FREESOUND_API_KEY": ""
      }
    }
  }
}
```

After PyPI, use [`examples/cursor.pypi.mcp.json`](examples/cursor.pypi.mcp.json) (`uvx` with args `["sonicmatch-mcp"]`). From a clone: `"command": "uv", "args": ["--directory", "/absolute/path/to/sonic-match-mcp", "run", "sonicmatch-mcp"]`. After `pip install`, `"command": "python3", "args": ["-m", "sonicmatch"]` works if that interpreter has the package.

## Codex

Codex reads `~/.codex/config.toml`. The same file is shared with the Codex IDE extension and the ChatGPT desktop app. Works today — [`examples/codex.config.toml`](examples/codex.config.toml):

```toml
[mcp_servers.sonicmatch]
command = "uvx"
args = ["--from", "git+https://github.com/js713-lab/sonic-match-mcp.git", "sonicmatch-mcp"]

[mcp_servers.sonicmatch.env]
GEMINI_API_KEY = ""
JAMENDO_CLIENT_ID = ""
FREESOUND_API_KEY = ""
```

Or from the CLI:

```bash
codex mcp add sonicmatch -- uvx --from git+https://github.com/js713-lab/sonic-match-mcp.git sonicmatch-mcp
```

After the PyPI release, the args shrink to `sonicmatch-mcp`. That file is [`examples/codex.pypi.config.toml`](examples/codex.pypi.config.toml):

```bash
codex mcp add sonicmatch -- uvx sonicmatch-mcp
```

`codex mcp list` should show `sonicmatch`. A trusted project can use `.codex/config.toml` instead of the home file.

User-owned Epidemic/Artlist JSON shape: [`examples/user_library.example.json`](examples/user_library.example.json). Registry metadata: [`server.json`](server.json). HTTP editors can point at `http://127.0.0.1:8765/mcp` after `sonicmatch-mcp --http`.

## Hosted (sonic.codecrafter.dev)

The GitHub repo stays **MIT**. Running it yourself is not metered.

The public endpoint charges credits per tool call. A trial key is 25 credits. Pack prices: Reel 500 credits / $9, Studio 3000 credits / $39. Card checkout is not connected on the host yet.

The landing page is https://sonic.codecrafter.dev. Use **Use the sample clip** there, or download [`examples/sample-clip.mp4`](examples/sample-clip.mp4) and upload it, to try a match without your own footage.

1. Open https://sonic.codecrafter.dev and take a trial key (or use a paid key).
2. `export SONICMATCH_API_KEY=sm_…`
3. Add to Cursor: the button on that page, or paste `examples/cursor.hosted.mcp.json` into `~/.cursor/mcp.json`.

```json
{
  "mcpServers": {
    "sonicmatch": {
      "url": "https://sonic.codecrafter.dev/mcp",
      "headers": {
        "Authorization": "Bearer ${env:SONICMATCH_API_KEY}"
      }
    }
  }
}
```

`--http` has no authentication. Keep it on loopback. The Docker image binds `0.0.0.0` so the container port works — do not publish that port to the internet. See [SECURITY.md](SECURITY.md).

## Architecture

```mermaid
flowchart TB
  subgraph mcp [MCP Server - FastMCP / Python - stdio + HTTP]
    tools[ingest_video / analyze_video_music / recommend_bgm / preview_mix / export_mix_spec / suggest_cuts]
  end
  tools --> ingest
  tools --> brain
  tools --> hub
  tools --> mixer
  ingest[Ingestor<br/>yt-dlp · ffmpeg · ffprobe · URL/file]
  brain[Video Brain<br/>Gemini / local VL · librosa · PySceneDetect · Whisper]
  hub[Music Hub<br/>seed CC · Jamendo · Freesound · user library · generate]
  mixer[Mixer<br/>ffmpeg · ducking · loop/trim · EDL cuts]
  hub --> index[Track index<br/>tags + license + embeddings · SQLite · optional LanceDB]
```

Hard rule: **never send raw multi-MB video through the MCP payload.** Store locally, pass an `asset_id`. Loopback, `file://`, and private IPs are rejected (SSRF).

## VideoSonic profile

Analysis returns structured JSON, not a paragraph:

```json
{
  "duration_sec": 18.4,
  "aspect": "9:16",
  "content_type": "lifestyle",
  "has_speech": true,
  "speech_coverage": 0.62,
  "existing_music": false,
  "overall_mood": ["warm", "playful"],
  "energy_mean": 0.62,
  "energy_curve": [{"t": 0, "energy": 0.3}, {"t": 4, "energy": 0.8}],
  "pacing": "fast-cut",
  "scenes": [{"start": 0, "end": 3.2, "description": "cafe exterior", "energy": 0.4}],
  "hook_window": [9.0, 15.0],
  "suggested_bpm": [95, 118],
  "avoid": ["dark cinematic drone", "aggressive trap", "lyrics-dense"],
  "search_queries": ["warm acoustic pop instrumental cafe"],
  "platform_hint": "instagram_reel",
  "analyzer": "local"
}
```

- **Primary:** Gemini video understanding when `GEMINI_API_KEY` is set.
- **Fallback:** ffmpeg scene cuts + WAV energy / silence / ZCR heuristics. Optional `faster-whisper`, `scenedetect`, `librosa` if installed (`pip install 'sonicmatch-mcp[local-vl]'`).

## Music hub

Pluggable, license-first. v0 ships:

| Adapter | When | License reality |
|---|---|---|
| **Seed catalog** (`data/seed_tracks.json`) | always | 20 CC0 / CC-BY Reel beds + a vocal fixture + a CC-BY-NC fixture (NC is never auto-recommended) |
| **Jamendo** | `JAMENDO_CLIENT_ID` | CC, check commercial |
| **Freesound** | `FREESOUND_API_KEY` | CC, good for beds/loops not songs |
| **User library JSON** | `SONICMATCH_LIBRARY_PATH` / `EPIDEMIC_LIBRARY_PATH` / `ARTLIST_LIBRARY_PATH` | **you** already licensed it; we do not scrape paid sites |
| **Generate** | `generate_bed` | always `source=generated`; local sine demo unless you swap a real model |

Ranking (weighted): mood/energy → instrumental if speech → duration/loop → BPM vs cut rate → license fit → tag embedding cosine → user constraints. `recommend_bgm` drops non-commercial and generated tracks instead of downranking them.

Tracks are indexed in SQLite (`~/.cache/sonicmatch-mcp/db/tracks.sqlite`) with a 24-d tag embedding. If `lancedb` is installed (`pip install 'sonicmatch-mcp[embeddings]'`), vectors are also upserted there.

Seed tracks have no remote audio files on purpose (you should host files you actually have the rights to). `preview_mix` synthesizes a CC0 demo bed so the mixer still runs offline. `generate_bed` is a catalog-miss fallback and is **not** cleared for ads.

## Docker

```bash
docker build -t sonicmatch-mcp .
# Loopback-only publish. The process inside the container has no HTTP auth.
docker run --rm -p 127.0.0.1:8765:8765 -v sonic-cache:/data/cache sonicmatch-mcp
```

## Roadmap

Catalog > new tools.

- [x] Freesound adapter (loops / beds)
- [x] Tag embeddings in SQLite (+ optional LanceDB extra)
- [x] Epidemic Sound / Artlist as **user-owned JSON** plugins (no scrape)
- [x] Beat-grid vs scene-cut suggestions (EDL-ish `suggest_cuts`)
- [x] MCP registry listing (`server.json`)
- [x] Generate tool, marked `source=generated` (local demo; swap a real model at your own legal risk)
- [x] Official MCP registry listing via GitHub Release MCPB (see [PUBLISH.md](PUBLISH.md))
- [x] Non-commercial licenses excluded from auto `recommend_bgm`
- [ ] 20 seed beds a Reel editor would actually keep, with audio you host
- [ ] User-owned Artlist / Epidemic JSON as the default path for ads
- [ ] Real CLAP audio embeddings
- [x] PyPI project metadata, console script, and tag-triggered Trusted Publishing (`.github/workflows/release.yml`)
- [ ] First upload on PyPI (`uvx sonicmatch-mcp`). Maintainer steps are in [PUBLISH.md](PUBLISH.md). The name was unregistered as of 2026-10-21.

## Why this can be a good open-source project

**Yes if** you nail: (1) video-native analysis, (2) license honesty on every row, (3) editor-shaped output (hook in/out, ducking, mix spec), (4) a catalog someone would keep.

**No if** you only wrap YouTube Music search, or if the first five recs sound like leftover stock beds.

Day-1 risk gates (enforced in code, not slogans):

| Risk | Gate |
|---|---|
| **Content ID** | Every rec/search/get_track includes `content_id_warning`. CC/RF is never "Content-ID-safe". `content_id_risk` is `unknown` or `likely`, never `cleared`. |
| **yt-dlp ToS / broken extractors** | Platform URL ingest is off unless `SONICMATCH_ALLOW_YTDLP=1`. Failures map to `YTDLP_EXTRACTOR` and tell you to pass a local file. |
| **Upload size / SSRF** | HTTPS-only remote ingest, no `file://` / loopback / private IPs, `SONICMATCH_MAX_DOWNLOAD_MB` (default 200) on files, HTTP, and yt-dlp `--max-filesize`. |
| **“Trending” is a closed Meta graph** | Queries for trending/viral/IG audio/TikTok sound return empty + `TRENDING_UNAVAILABLE`. `recommend_bgm` always sets `trending_available=false`. |
| **Generation-model commercial terms** | `generate_bed` refuses unless `i_understand_not_commercially_cleared=true`. Generated tracks are excluded from auto `recommend_bgm`. |

## Use cases

IG Reel / Story · Shopee product clip · YouTube Shorts agent · CapCut/Premiere companion · campus recap · podcast clipper · travel-vlog batch · brand-kit lock (BPM + no vocals) · silent-film / accessibility · multi-agent studio.

## Contributing

[CONTRIBUTING.md](CONTRIBUTING.md) · [CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md) · [SECURITY.md](SECURITY.md) · [CHANGELOG.md](CHANGELOG.md) · [starter issues](docs/good-first-issues.md)

## License

MIT. Track licenses are independent of the repo license.

From [CodeCrafter](https://codecrafter.dev).
