Metadata-Version: 2.4
Name: dhcckb-mcp-daizhigev20
Version: 0.3.0
Summary: MCP Server for searching and processing ancient Chinese TXT texts from the daizhigev20 GitHub repository. v0.3.0: 修复 GitHub 中文路径下载失败，Contents API 优先 + Raw URL 回退 + Blob API 兜底，选个编码防双重编码，404 不重试，新增 inspect_book_source 调试工具。
Author: Digital Humanities Platform
License: MIT
License-File: LICENSE
Keywords: ancient-chinese,digital-humanities,mcp,text-analysis,txt
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Education
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Classifier: Topic :: Text Processing
Requires-Python: >=3.12
Requires-Dist: charset-normalizer>=3.0.0
Requires-Dist: mcp>=1.0.0
Requires-Dist: opencc-python-reimplemented>=0.1.7
Requires-Dist: platformdirs>=4.0.0
Requires-Dist: requests>=2.31.0
Provides-Extra: dev
Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff>=0.5; extra == 'dev'
Provides-Extra: test
Requires-Dist: pytest-asyncio>=0.23; extra == 'test'
Requires-Dist: pytest>=8.0; extra == 'test'
Description-Content-Type: text/markdown

# mcp-daizhigev20

MCP Server for searching and processing ancient Chinese TXT texts from the [daizhigev20](https://github.com/garychowcmu/daizhigev20) GitHub repository.

## Overview

`mcp-daizhigev20` is a Python-based MCP (Model Context Protocol) server that provides:

- **Index Building**: Enumerate TXT files from a GitHub repository and build a local SQLite index with OpenCC simplified/traditional Chinese conversion
- **Book Search**: Multi-layer matching (exact, normalized, prefix, fuzzy) with simplified/traditional Chinese compatibility
- **Document Inspection**: Comprehensive structural analysis including encoding detection, character analysis, heading detection, and bracket analysis
- **Format Conversion**: Convert ancient Chinese texts to structured JSON or RAG-ready JSONL format

## Installation

```bash
pip install mcp-daizhigev20
```

Or via uvx:

```bash
uvx mcp-daizhigev20
```

### Requirements

- Python >= 3.12
- OpenCC data (included via `opencc-python-reimplemented`)

## Quick Start

1. **Build the index** (required first step):

```
MCP tool: build_index
```

This enumerates all TXT files in the daizhigev20 repository and builds a local SQLite index.

2. **Search for a book**:

```
MCP tool: search_books(query="资治通鉴")
```

3. **Select a book** from search results:

```
MCP tool: select_book(search_id="...", rank=1)
```

4. **Inspect the document** structure:

```
MCP tool: inspect_book(book_id="...")
```

5. **Convert** to your desired format:

```
MCP tool: convert_book(book_id="...", output_format="jsonl")
```

## Tools

| Tool | Description |
|------|-------------|
| `build_index` | Build/rebuild the local SQLite book index |
| `search_books` | Search for books with multi-layer matching |
| `select_book` | Select a book from search results |
| `inspect_book` | Comprehensive document structure analysis |
| `convert_book` | Convert to structured JSON or RAG JSONL |

## Configuration

### Environment Variables

| Variable | Description | Default |
|----------|-------------|---------|
| `GITHUB_TOKEN` | GitHub personal access token (optional) | None |
| `DAIZHIGE_DATA_DIR` | Base data directory | `~/.local/share/mcp-daizhigev20/` |
| `DAIZHIGE_DB_PATH` | SQLite database path | `<data_dir>/index.db` |
| `DAIZHIGE_CACHE_DIR` | Download cache directory | `<data_dir>/cache/` |
| `DAIZHIGE_ARTIFACT_DIR` | Artifact output directory | `<data_dir>/artifacts/` |
| `DAIZHIGE_TMP_DIR` | Temporary files directory | `<data_dir>/tmp/` |

### Automated Index Updates

Use external schedulers to keep the index up to date:

**cron** (every 6 hours):
```
0 */6 * * * daizhige-mcp-index
```

**systemd timer**: Create a service that calls `build_index` via MCP client.

> Note: scheduled index updates are not part of the 0.1.0 server itself — use cron, systemd timer, or CI pipelines.

## Output Formats

### Structured JSON (`output_format="json"`)

Preserves volumes, chapters, sections, paragraphs, and annotations with full source metadata.

```json
{
  "source": { "repository": "...", "git_sha": "..." },
  "metadata": { "title_original": "..." },
  "sections": [...],
  "paragraphs": [...]
}
```

### RAG JSONL (`output_format="jsonl"`)

Structure-aware chunking with configurable `chunk_size` (default 1200) and `chunk_overlap` (default 150). Chunks respect section boundaries and use paragraph/sentence-aware splitting.

```jsonl
{"id": "chunk_000001", "text": "...", "metadata": {...}}
{"id": "chunk_000002", "text": "...", "metadata": {...}}
```

## Chunking Strategy

- Priority 1: Respect volume/chapter/section boundaries
- Priority 2: Split at paragraph boundaries
- Priority 3: Split at sentence punctuation (。！？；)
- Fallback: Safe window (400 chars) for unpunctuated classical text

## Future Plans (not in 0.1.0)

- TEI/XML output format
- Markdown output
- EPUB output
- IIIF Annotation export
- CSV/database import format

## License

MIT