Metadata-Version: 2.4
Name: agent-video-downloader
Version: 1.2.1
Summary: yt-dlp for agents: a cloud-CLI multi-agent video downloader for TikTok, Instagram, Douyin, Rednote, Reddit, X.com
Author-email: ansaribilal1402 <noreply@github.com>
Maintainer-email: ansaribilal1402 <noreply@github.com>
License: MIT
Project-URL: Homepage, https://github.com/hamza140202/agent-video-downloader
Project-URL: Repository, https://github.com/hamza140202/agent-video-downloader
Project-URL: Documentation, https://github.com/hamza140202/agent-video-downloader#readme
Project-URL: Bug-Tracker, https://github.com/hamza140202/agent-video-downloader/issues
Project-URL: Changelog, https://github.com/hamza140202/agent-video-downloader/releases
Keywords: video,downloader,tiktok,instagram,douyin,rednote,xiaohongshu,reddit,twitter,agents,cli,mcp
Classifier: Development Status :: 5 - Production/Stable
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Internet :: WWW/HTTP
Classifier: Topic :: Multimedia :: Video
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: httpx<1.0,>=0.27
Requires-Dist: aiofiles>=23.0
Requires-Dist: pydantic<3.0,>=2.7
Requires-Dist: pydantic-settings>=2.3
Requires-Dist: tenacity<10.0,>=8.5
Requires-Dist: pybreaker<2.0,>=1.3
Requires-Dist: rich<14.0,>=13.0
Requires-Dist: click<9.0,>=8.1
Requires-Dist: python-magic>=0.4.27
Requires-Dist: structlog>=24.0
Requires-Dist: aiosqlite>=0.20
Requires-Dist: beautifulsoup4>=4.12
Requires-Dist: lxml>=5.2
Requires-Dist: Levenshtein>=0.25
Requires-Dist: yt-dlp>=2025.10.0
Requires-Dist: feedparser>=6.0
Provides-Extra: browser
Requires-Dist: playwright>=1.40; extra == "browser"
Provides-Extra: http
Requires-Dist: uvicorn>=0.30; extra == "http"
Requires-Dist: fastapi>=0.110; extra == "http"
Provides-Extra: mcp
Requires-Dist: mcp>=1.0; extra == "mcp"
Provides-Extra: socks
Requires-Dist: socksio>=1.0; extra == "socks"
Provides-Extra: dev
Requires-Dist: pytest<10.0,>=8.0; extra == "dev"
Requires-Dist: pytest-asyncio>=0.23; extra == "dev"
Requires-Dist: pytest-cov>=5.0; extra == "dev"
Requires-Dist: pytest-mock>=3.12; extra == "dev"
Requires-Dist: respx>=0.21; extra == "dev"
Requires-Dist: hypothesis>=6.100; extra == "dev"
Requires-Dist: freezegun>=1.5; extra == "dev"
Dynamic: license-file

# avd — agent-video-downloader

> **yt-dlp for cloud AI agents.** One command downloads videos from **TikTok, Instagram, Douyin, Rednote (Xiaohongshu), Reddit, X.com** — no browser, no cookies, no login. Built for Claude / Cursor / Cline / GLM / GPT agents that need video bytes in task workflows.

[![CI](https://github.com/hamza140202/agent-video-downloader/actions/workflows/ci.yml/badge.svg)](https://github.com/hamza140202/agent-video-downloader/actions)
[![PyPI version](https://img.shields.io/pypi/v/agent-video-downloader.svg)](https://pypi.org/project/agent-video-downloader/)
[![PyPI downloads](https://img.shields.io/pypi/dm/agent-video-downloader.svg)](https://pypi.org/project/agent-video-downloader/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)

**Install:**
```bash
pip install agent-video-downloader
avd agent-setup
```

**Live on PyPI:** https://pypi.org/project/agent-video-downloader/

---

## Quickstart — one command for AI agents

```bash
pip install agent-video-downloader
avd agent-setup
avd download 'https://www.tiktok.com/@scout2015/video/6718335390845095173'
```

`avd agent-setup` auto-installs ffmpeg, all Python deps, and the XHS-Downloader repo (for Rednote). It's idempotent — safe to re-run anytime. **Zero manual steps for an AI agent.**

For the full agent usage guide (8 steps, decision tree, MCP server config, env vars):
```bash
avd agent-instructions
```

---

## Why

`yt-dlp` is broken from datacenter IPs for TikTok, Twitter, Reddit, Instagram, Douyin. Cloud CLI agents (Claude Code, GLM CLI, Cursor, Cline) that need to download videos for tasks (transcription, OCR, content analysis, archival) have no reliable path. `avd` fixes that with **direct-API fallback chains** that work from cloud IPs without login.

All six platforms download real video bytes (verified live 2026-10-03 from a Hong Kong datacenter IP, total 1.2 GB downloaded through `avd` itself):

| Platform | Method | Sample download |
|---|---|---|
| **TikTok** | TikWM mirror API | 2.0 MB / 10.5 s |
| **Twitter/X** | api.fxtwitter.com → video.twimg.com | 156 MB / 74.8 s 4K |
| **Reddit** | rapidsave.com + v.redd.it CMAF/DASH + ffmpeg mux | 6.1 MB / 16.1 s |
| **Instagram** | yt-dlp + facebookexternalhit/1.1 UA | 8.4 MB / 77.3 s |
| **Douyin** | api.douyin.wtf public demo (zero-config) | 3.6 MB / 27.6 s |
| **Rednote (XHS)** | XHS-Downloader (curl_cffi chrome146) | 8.4 MB / 720p |

### Babymonster batch test — 21/24 videos downloaded via `avd` itself

24 real K-pop interview/documentary URLs (4 per platform minimum):
- TikTok: 4/4 ✅ — Babymonster YG official countdown interviews
- Twitter: 4/4 ✅ — SpaceX 4K static fire (548 MB total)
- Instagram: 4/4 ✅ — Babymonster reels (All Night Nippon podcast)
- Reddit: 4/4 ✅ — Babymonster interviews incl. 6-min 372s video (468 MB total)
- Douyin: 4/4 ✅ — Babymonster/Ahyeon choreography (152 MB total)
- Rednote: 1/4 ⚠️ — 1 verified image; 3 bot-walled without cookie (documented limitation)

Total: **1.2 GB of real K-pop content** downloaded through `avd` itself.

---

## Commands

| Command | Purpose |
|---|---|
| `avd agent-setup` | One-command bootstrap — installs ffmpeg, deps, XHS-Downloader |
| `avd agent-instructions` | Print step-by-step usage guide for AI agents |
| `avd download <url>` | Download one URL |
| `avd batch <file>` | Batch download (one URL per line, `#` comments OK) |
| `avd verify <path>` | Verify a downloaded file's integrity |
| `avd test --smoke` | End-to-end self-test on one URL per platform |
| `avd jobs` | List recent jobs |
| `avd dlq` | List dead-letter queue entries |
| `avd replay <dlq_id>` | Replay a failed job |
| `avd mcp` | Start MCP server (stdio JSON-RPC 2.0) |
| `avd supported` | Show supported platforms |
| `avd agents` | Show agent versions |

---

## Install

### From PyPI (recommended)

```bash
pip install agent-video-downloader
avd agent-setup
```

### From source (development)

```bash
git clone https://github.com/hamza140202/agent-video-downloader
cd agent-video-downloader
pip install -e .[dev]
avd agent-setup
```

### System requirements

- Python 3.10+ (tested on 3.10, 3.11, 3.12, 3.13)
- `ffmpeg` / `ffprobe` (auto-installed by `avd agent-setup` on Debian/Ubuntu; manual on other systems)
- `git` (for cloning XHS-Downloader on first Rednote download)

---

## Configuration (env vars)

| Var | Default | Description |
|---|---|---|
| `AVD_DOWNLOAD_DIR` | `./download` | Default download destination |
| `AVD_LOG_LEVEL` | `INFO` | `DEBUG`/`INFO`/`WARNING`/`ERROR` |
| `AVD_REDDIT_CLIENT_ID` | — | Reddit OAuth (only needed if rapidsave.com fails — rare) |
| `AVD_REDDIT_CLIENT_SECRET` | — | same |
| `AVD_REDDIT_USERNAME` | — | throwaway Reddit username |
| `AVD_REDDIT_PASSWORD` | — | throwaway Reddit password |
| `AVD_DOUYIN_DTK_URL` | — | self-hosted Evil0ctal DTK (optional — defaults to public demo) |
| `AVD_XHS_DOWNLOADER_PATH` | `~/XHS-Downloader` | override XHS-Downloader clone path |
| `AVD_XHS_COOKIE` | — | XHS cookie (optional, unlocks HD Rednote videos) |
| `AVD_PROXY` | — | SOCKS5/HTTP proxy URL for all requests |

---

## Architecture (4 plain-Python agents, no LLM in runtime loop)

```
URL → Orchestrator → [TikTok|Twitter|Reddit|Instagram|Rednote|Douyin] extractors (fallback chain)
                     → Verifier (6-layer: size, magic bytes, ffprobe, duration, streams, moov)
                     → Truth Agent (cross-check vs source platform: oEmbed / syndication / RSS / OG)
                     → verified MP4 + manifest
```

- **Orchestrator** — owns the extractor registry, drives the fallback chain, persists job state to SQLite
- **Verifier** — 6-layer integrity check (size + magic bytes + ffprobe + duration + streams + moov atom)
- **Truth Agent** — cross-references downloaded metadata against source platform
- **Tester** — runs end-to-end on `tests/sample_urls.json`

Per-platform fallback chains (slot-based state machine, ytagent doctrine):
- **TikTok**: TikWM → embed/v2 → tiklydown → oEmbed
- **Twitter/X**: fxtwitter → syndication → unrollnow → vxtwitter
- **Reddit**: rapidsave.com/info + v.redd.it CMAF/DASH direct + ffmpeg mux → rapidsave server-side mux → yt-dlp+OAuth → RSS image
- **Instagram**: yt-dlp+facebookexternalhit UA → embed/captioned+fb-UA → embed+android-UA → embed+ios-UA → ddinstagram mirror
- **Douyin**: api.douyin.wtf demo → self-hosted DTK sidecar → yt-dlp
- **Rednote**: XHS-Downloader (curl_cffi) → curl_cffi direct → yt-dlp

Full design: [`docs/architecture.md`](docs/architecture.md) · Full research: [`docs/research-report.md`](docs/research-report.md)

---

## Documentation

### For users
- [`README.md`](README.md) — this file (install + quickstart + commands)
- Run `avd agent-instructions` — interactive step-by-step usage guide

### For developers / agents modifying the code
- [`CLAUDE.md`](CLAUDE.md) — project memory (read first if modifying)
- [`AGENTS.md`](AGENTS.md) — agent contracts + perfection prompting rules
- [`TECHSTACK.md`](TECHSTACK.md) — every dependency, every version, why
- [`PHASES.md`](PHASES.md) — development phases with exit criteria
- [`PLAN.md`](PLAN.md) — concrete execution plan with task IDs
- [`SKILLS.md`](SKILLS.md) — per-agent skill spec sheets

### Deep technical references
- [`docs/research-report.md`](docs/research-report.md) — live-verified endpoint research per platform
- [`docs/architecture.md`](docs/architecture.md) — system design with ASCII diagram
- [`docs/endpoint-matrix.md`](docs/endpoint-matrix.md) — living endpoint table

---

## License

MIT. See [LICENSE](LICENSE).

## Repo

- Primary: https://github.com/hamza140202/agent-video-downloader
- PyPI: https://pypi.org/project/agent-video-downloader/
- Issues: file on GitHub with the `metadata.extractor_chain` and `metadata.slots_tried` from your run's manifest attached

## Changelog

### v1.2.0 (2026-10-03) — PyPI release
- One-command install: `pip install agent-video-downloader && avd agent-setup`
- New `avd agent-setup` command (auto-install ffmpeg, deps, XHS-Downloader)
- New `avd agent-instructions` command (8-step AI agent usage guide)
- Auto-bootstrap on first Rednote download (no manual setup)
- Reddit extractor fix: probe BOTH CMAF and DASH format ladders
- Rednote extractor fix: parse XHS-Downloader stdout for actual success
- Verifier: adaptive min_size_bytes (50KB images / 5KB audio / 1MB video)
- Verifier: skip slow integrity_decode for files > 50 MB
- Babymonster batch test: 21/24 videos downloaded via `avd` itself (1.2 GB)
- Published to PyPI: https://pypi.org/project/agent-video-downloader/1.2.0/

### v1.1.0 (2026-10-03) — all 6 platforms real downloads
- Replaced honest-empties with verified-working methods per ytagent doctrine
- TikTok via TikWM (2 MB MP4 verified)
- Twitter via fxtwitter (156 MB 4K MP4 verified)
- Reddit via rapidsave.com + v.redd.it CMAF + ffmpeg mux (6.1 MB verified)
- Instagram via yt-dlp + facebookexternalhit UA (8.4 MB verified)
- Douyin via api.douyin.wtf public demo (3.6 MB verified)
- Rednote via XHS-Downloader (curl_cffi) (8.4 MB verified)
- Truth Agent: Twitter returned `verified` verdict

### v1.0.0 (2026-10-03) — initial release
- Multi-agent architecture (Orchestrator + Verifier + Truth + Tester)
- 6 platform extractors with fallback chains
- MCP server (stdio JSON-RPC 2.0)
- SQLite jobs table with resume / DLQ replay
- 30/30 unit tests pass
