Metadata-Version: 2.4
Name: duizhang
Version: 0.1.0
Summary: LLM Chinese Couplet Generation Evaluation Framework
Author: DuiZhang Contributors
License-Expression: GPL-3.0-or-later
Project-URL: Homepage, https://github.com/cycleuser/DuiZhang
Project-URL: Repository, https://github.com/cycleuser/DuiZhang
Project-URL: Issues, https://github.com/cycleuser/DuiZhang/issues
Keywords: ollama,couplet,chinese,nlp,evaluation,llm,rag,cli
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Environment :: Web Environment
Classifier: Framework :: Flask
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Text Processing :: Linguistic
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: ollama>=0.1.16
Requires-Dist: numpy>=1.20.0
Requires-Dist: pandas>=1.3.0
Requires-Dist: tqdm>=4.62.0
Requires-Dist: scikit-learn>=1.0.0
Requires-Dist: jieba>=0.42.1
Requires-Dist: matplotlib>=3.5.0
Requires-Dist: pypinyin>=0.47.0
Requires-Dist: pypdf>=3.0.0
Requires-Dist: langchain>=0.1.0
Requires-Dist: langchain-community>=0.0.10
Requires-Dist: faiss-cpu>=1.7.4
Requires-Dist: gensim>=4.3.0
Requires-Dist: flask>=3.0
Requires-Dist: flask-cors>=4.0
Requires-Dist: rich>=13.0
Requires-Dist: prompt_toolkit>=3.0
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0; extra == "dev"
Requires-Dist: ruff>=0.1.0; extra == "dev"
Dynamic: license-file

# DuiZhang (对仗)

[![PyPI version](https://img.shields.io/pypi/v/duizhang.svg)](https://pypi.org/project/duizhang/)
[![Python](https://img.shields.io/pypi/pyversions/duizhang.svg)](https://pypi.org/project/duizhang/)
[![License: GPL-3.0](https://img.shields.io/badge/License-GPL%203.0-blue.svg)](LICENSE)

**LLM Chinese Couplet Generation Evaluation Framework** — Evaluate and compare large language models' Chinese language capabilities through traditional couplet (对联) generation tasks.

## Features

- **Rule-based Evaluation**: 6 core metrics (POS matching, structure, rhythm, tone, content relevance, imagery correspondence) + 2 base metrics (length match, no duplicate)
- **LLM Self-Evaluation**: Model self-assessment across all 6 dimensions for meta-cognitive analysis
- **PDF RAG Pipeline**: Build knowledge bases from academic PDFs and generate literature reviews
- **Multi-Model Comparison**: Evaluate multiple Ollama models side by side with radar chart visualization
- **CLI & Python API**: Full command-line interface and programmatic access
- **Offline Design**: All processing runs locally via Ollama — no external API calls required

## Requirements

- Python 3.10+
- [Ollama](https://ollama.ai/) running locally
- Chat model (e.g., `qwen2.5:7b`)
- Embedding model (e.g., `nomic-embed-text`)

## Installation

```bash
# From PyPI
pip install duizhang

# From source
git clone https://github.com/cycleuser/DuiZhang.git
cd DuiZhang
pip install -e .
```

## Quick Start

```bash
# Start Ollama
ollama serve
ollama pull qwen2.5:7b
ollama pull nomic-embed-text

# Evaluate a model
duizhang eval --model qwen2.5:7b --samples 10

# Quick generation
duizhang run --model qwen2.5:7b --input "春风送暖入屠苏"

# Process PDFs
duizhang pdf --force-rebuild

# List available models
duizhang models

# Show configuration
duizhang config show
```

## Python API

```python
from duizhang import evaluate_couplet, index_documents, ToolResult

# Evaluate a single couplet
result = evaluate_couplet(
    input_line="春风送暖",
    expected="冬雪飘香",
    generated="秋月寒江",
)
print(result.metrics)

# Index PDF documents
result = index_documents(["paper1.pdf", "paper2.pdf"])
```

## Evaluation Metrics

### Rule-based Metrics

| Metric | Range | Description |
|--------|-------|-------------|
| POS Match | 0-1 | Part-of-speech correspondence between lines |
| Structure Match | 0-1 | Punctuation position alignment |
| Rhythm Match | 0-1 | Word segmentation length pattern matching |
| Tone Match | 0-1 | Ping/Ze (平仄) tone opposition |
| Content Relevance | 0-1 | TF-IDF semantic similarity |
| Imagery Correspondence | 0-1 | Noun quantity and reflection matching |

### Base Metrics

| Metric | Range | Description |
|--------|-------|-------------|
| Length Match | 0-1 | Generated vs expected length alignment |
| No Duplicate | 0-1 | Character non-repetition between lines |

### LLM Self-Evaluation

The model evaluates its own output across all 6 rule-based dimensions, enabling meta-cognitive analysis by comparing algorithm scores vs self-assessment scores.

## Project Structure

```
DuiZhang/
├── duizhang/
│   ├── __init__.py          # Version & public API
│   ├── __main__.py          # python -m duizhang entry
│   ├── api.py               # Unified Python API
│   ├── cli.py               # Entry point routing
│   ├── cli_app.py           # CLI application
│   ├── tools.py             # OpenAI function-calling tools
│   ├── core/
│   │   ├── __init__.py      # Core module exports
│   │   ├── config.py        # Configuration dataclass
│   │   ├── constants.py     # Application constants
│   │   ├── errors.py        # Custom exception hierarchy
│   │   ├── ollama_client.py # Ollama API client
│   │   ├── pdf_processor.py # PDF text extraction
│   │   ├── kb_builder.py    # FAISS knowledge base builder
│   │   └── summarizer.py    # Document summarization
│   ├── evaluator/
│   │   ├── __init__.py
│   │   ├── metrics.py       # Rule-based metrics
│   │   ├── llm_metrics.py   # LLM self-evaluation
│   │   ├── data_loader.py   # Couplet data loading
│   │   ├── evaluator.py     # Main evaluation engine
│   │   └── visualizer.py    # Radar chart visualization
│   ├── pdf_rag/
│   │   ├── __init__.py
│   │   ├── pipeline.py      # PDF processing pipeline
│   │   └── report_generator.py
│   ├── templates/           # Web UI templates
│   └── static/              # CSS/JS assets
├── tests/                   # Test suite
├── data/                    # Runtime data (auto-created)
└── images/                  # Screenshots
```

## Testing

```bash
pip install -e ".[dev]"
pytest -v
pytest -v --cov=duizhang
```

## License

GPL-3.0-or-later. See [LICENSE](LICENSE) for details.
