Metadata-Version: 2.4
Name: agent-video-downloader
Version: 1.2.0
Summary: yt-dlp for agents: a cloud-CLI multi-agent video downloader for TikTok, Instagram, Douyin, Rednote, Reddit, X.com
Author-email: Bilal140202 <noreply@github.com>
License: MIT
Project-URL: Homepage, https://github.com/hamza140202/agent-video-downloader
Project-URL: Repository, https://github.com/hamza140202/agent-video-downloader
Project-URL: Documentation, https://github.com/hamza140202/agent-video-downloader#readme
Project-URL: Bug-Tracker, https://github.com/hamza140202/agent-video-downloader/issues
Project-URL: Changelog, https://github.com/hamza140202/agent-video-downloader/releases
Keywords: video,downloader,tiktok,instagram,douyin,rednote,xiaohongshu,reddit,twitter,agents,cli,mcp
Classifier: Development Status :: 5 - Production/Stable
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Internet :: WWW/HTTP
Classifier: Topic :: Multimedia :: Video
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: httpx<1.0,>=0.27
Requires-Dist: aiofiles>=23.0
Requires-Dist: pydantic<3.0,>=2.7
Requires-Dist: pydantic-settings>=2.3
Requires-Dist: tenacity<10.0,>=8.5
Requires-Dist: pybreaker<2.0,>=1.3
Requires-Dist: rich<14.0,>=13.0
Requires-Dist: click<9.0,>=8.1
Requires-Dist: python-magic>=0.4.27
Requires-Dist: structlog>=24.0
Requires-Dist: aiosqlite>=0.20
Requires-Dist: beautifulsoup4>=4.12
Requires-Dist: lxml>=5.2
Requires-Dist: Levenshtein>=0.25
Requires-Dist: yt-dlp>=2025.10.0
Requires-Dist: feedparser>=6.0
Provides-Extra: browser
Requires-Dist: playwright>=1.40; extra == "browser"
Provides-Extra: http
Requires-Dist: uvicorn>=0.30; extra == "http"
Requires-Dist: fastapi>=0.110; extra == "http"
Provides-Extra: mcp
Requires-Dist: mcp>=1.0; extra == "mcp"
Provides-Extra: socks
Requires-Dist: socksio>=1.0; extra == "socks"
Provides-Extra: dev
Requires-Dist: pytest<10.0,>=8.0; extra == "dev"
Requires-Dist: pytest-asyncio>=0.23; extra == "dev"
Requires-Dist: pytest-cov>=5.0; extra == "dev"
Requires-Dist: pytest-mock>=3.12; extra == "dev"
Requires-Dist: respx>=0.21; extra == "dev"
Requires-Dist: hypothesis>=6.100; extra == "dev"
Requires-Dist: freezegun>=1.5; extra == "dev"
Dynamic: license-file

# avd — agent-video-downloader

> **yt-dlp for cloud AI agents.** One command downloads videos from **TikTok, Instagram, Douyin, Rednote (Xiaohongshu), Reddit, X.com** — no browser, no cookies, no login. Built for Claude / Cursor / Cline / GLM / GPT agents that need video bytes in task workflows.

[![CI](https://github.com/hamza140202/agent-video-downloader/actions/workflows/ci.yml/badge.svg)](https://github.com/hamza140202/agent-video-downloader/actions)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
[![PyPI version](https://img.shields.io/pypi/v/agent-video-downloader.svg)](https://pypi.org/project/agent-video-downloader/)

---

## Quickstart — one command for AI agents

```bash
pip install agent-video-downloader
avd agent-setup
avd download 'https://www.tiktok.com/@scout2015/video/6718335390845095173'
```

`avd agent-setup` auto-installs ffmpeg, all Python deps, and the XHS-Downloader repo (for Rednote). It's idempotent — safe to re-run anytime.

For the full agent usage guide:
```bash
avd agent-instructions
```

---

## Why

`yt-dlp` is broken from datacenter IPs for TikTok, Twitter, Reddit, Instagram, Douyin. Cloud CLI agents (Claude Code, GLM CLI, Cursor, Cline) that need to download videos for tasks (transcription, OCR, content analysis, archival) have no reliable path. `avd` fixes that with **direct-API fallback chains** that work from cloud IPs without login.

All six platforms download real video bytes (verified live 2026-10-03 from a Hong Kong datacenter IP):

| Platform | Method | Sample download |
|---|---|---|
| **TikTok** | TikWM mirror API | 2.0 MB / 10.5 s |
| **Twitter/X** | api.fxtwitter.com → video.twimg.com | 156 MB / 74.8 s 4K |
| **Reddit** | rapidsave.com + v.redd.it CMAF + ffmpeg mux | 6.1 MB / 16.1 s |
| **Instagram** | yt-dlp + facebookexternalhit/1.1 UA | 8.4 MB / 77.3 s |
| **Douyin** | api.douyin.wtf public demo (zero-config) | 3.6 MB / 27.6 s |
| **Rednote (XHS)** | XHS-Downloader (curl_cffi chrome146) | 8.4 MB / 720p |

---

## Commands

| Command | Purpose |
|---|---|
| `avd agent-setup` | One-command bootstrap — installs ffmpeg, deps, XHS-Downloader |
| `avd agent-instructions` | Print step-by-step usage guide for AI agents |
| `avd download <url>` | Download one URL |
| `avd batch <file>` | Batch download (one URL per line, `#` comments OK) |
| `avd verify <path>` | Verify a downloaded file's integrity |
| `avd test --smoke` | End-to-end self-test on one URL per platform |
| `avd jobs` | List recent jobs |
| `avd dlq` | List dead-letter queue entries |
| `avd replay <dlq_id>` | Replay a failed job |
| `avd mcp` | Start MCP server (stdio JSON-RPC 2.0) |
| `avd supported` | Show supported platforms |
| `avd agents` | Show agent versions |

---

## Install

### From PyPI (recommended)

```bash
pip install agent-video-downloader
avd agent-setup
```

### From source (development)

```bash
git clone https://github.com/hamza140202/agent-video-downloader
cd agent-video-downloader
pip install -e .[dev]
avd agent-setup
```

### System requirements

- Python 3.10+ (tested on 3.10, 3.11, 3.12, 3.13)
- `ffmpeg` / `ffprobe` (auto-installed by `avd agent-setup` on Debian/Ubuntu; manual on other systems)
- `git` (for cloning XHS-Downloader on first Rednote download)

---

## Configuration (env vars)

| Var | Default | Description |
|---|---|---|
| `AVD_DOWNLOAD_DIR` | `./download` | Default download destination |
| `AVD_LOG_LEVEL` | `INFO` | `DEBUG`/`INFO`/`WARNING`/`ERROR` |
| `AVD_REDDIT_CLIENT_ID` | — | Reddit OAuth (only needed if rapidsave.com fails) |
| `AVD_REDDIT_CLIENT_SECRET` | — | same |
| `AVD_REDDIT_USERNAME` | — | throwaway Reddit username |
| `AVD_REDDIT_PASSWORD` | — | throwaway Reddit password |
| `AVD_DOUYIN_DTK_URL` | — | self-hosted Evil0ctal DTK (optional — defaults to public demo) |
| `AVD_XHS_DOWNLOADER_PATH` | `~/XHS-Downloader` | override XHS-Downloader clone path |
| `AVD_PROXY` | — | SOCKS5/HTTP proxy URL for all requests |

---

## Architecture (4 plain-Python agents, no LLM in runtime loop)

```
URL → Orchestrator → [TikTok|Twitter|Reddit|Instagram|Rednote|Douyin] extractors (fallback chain)
                     → Verifier (6-layer: size, magic bytes, ffprobe, duration, streams, moov)
                     → Truth Agent (cross-check vs source platform: oEmbed / syndication / RSS / OG)
                     → verified MP4 + manifest
```

- **Orchestrator** — owns the extractor registry, drives the fallback chain, persists job state to SQLite
- **Verifier** — 6-layer integrity check (size + magic bytes + ffprobe + duration + streams + moov atom)
- **Truth Agent** — cross-references downloaded metadata against source platform
- **Tester** — runs end-to-end on `tests/sample_urls.json`

Per-platform fallback chains (slot-based state machine, ytagent doctrine):
- **TikTok**: TikWM → embed/v2 → tiklydown → oEmbed
- **Twitter/X**: fxtwitter → syndication → unrollnow → vxtwitter
- **Reddit**: rapidsave.com/info + v.redd.it CMAF direct + ffmpeg mux → rapidsave server-side mux → yt-dlp+OAuth → RSS image
- **Instagram**: yt-dlp+facebookexternalhit UA → embed/captioned+fb-UA → embed+android-UA → embed+ios-UA → ddinstagram mirror
- **Douyin**: api.douyin.wtf demo → self-hosted DTK sidecar → yt-dlp
- **Rednote**: XHS-Downloader (curl_cffi) → curl_cffi direct → yt-dlp

Full design: [`docs/architecture.md`](docs/architecture.md) · Full research: [`docs/research-report.md`](docs/research-report.md)

---

## Documentation

- [`CLAUDE.md`](CLAUDE.md) — project memory (read first if modifying)
- [`AGENTS.md`](AGENTS.md) — agent contracts + perfection prompting rules
- [`TECHSTACK.md`](TECHSTACK.md) — every dependency, every version, why
- [`PHASES.md`](PHASES.md) — development phases with exit criteria
- [`PLAN.md`](PLAN.md) — concrete execution plan with task IDs
- [`SKILLS.md`](SKILLS.md) — per-agent skill spec sheets
- [`docs/research-report.md`](docs/research-report.md) — live-verified endpoint research per platform
- [`docs/architecture.md`](docs/architecture.md) — system design with ASCII diagram
- [`docs/endpoint-matrix.md`](docs/endpoint-matrix.md) — living endpoint table

---

## License

MIT. See [LICENSE](LICENSE).

## Repo

- Primary: https://github.com/hamza140202/agent-video-downloader
- Issues: file on GitHub with the `metadata.extractor_chain` and `metadata.slots_tried` from your run's manifest attached
