Metadata-Version: 2.4
Name: modularragagain
Version: 0.1.0
Summary: Modular RAG pipeline on LangChain + DashScope: query rewrite, HyDE, hybrid retrieval with RRF, rerank, compression and long-context reordering.
Keywords: rag,retrieval-augmented-generation,langchain,dashscope,chroma,rerank,hyde
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Operating System :: OS Independent
Classifier: Intended Audience :: Developers
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Text Processing :: Linguistic
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: jieba>=0.42.1
Requires-Dist: python-dotenv>=1.0
Requires-Dist: langchain-core>=1.0
Requires-Dist: langchain-openai>=1.0
Requires-Dist: langchain-community>=0.4
Requires-Dist: langchain-chroma>=1.0
Requires-Dist: langchain-classic>=1.0
Requires-Dist: chromadb>=1.0
Requires-Dist: dashscope>=1.20
Provides-Extra: dev
Requires-Dist: build>=1.2; extra == "dev"
Requires-Dist: twine>=5.0; extra == "dev"

# modularRAGagain

基于 **LangChain + 阿里云 DashScope** 的模块化 RAG（检索增强生成）流水线，把检索链路拆成 11 个可组合模块，按需拼装即可。

## 特性

| # | 模块 | 说明 |
|---|------|------|
| 1 | 查询规范化 | 去多余空白、统一格式 |
| 2 | 查询改写 | 将多轮/口语化问题改写为独立检索查询 |
| 3 | HyDE | 生成假设性答案做语义检索 |
| 4 | 向量检索 | Chroma + DashScope Embeddings |
| 5 | BM25 检索 | jieba 中文分词 + BM25 |
| 6 | 混合检索 | EnsembleRetriever + RRF 融合 |
| 7 | 结果去重 | 按正文去重 |
| 8 | 重排序 | DashScope `qwen3-rerank` 精排 |
| 9 | 相似度过滤 | 余弦相似度阈值过滤并写回 metadata |
| 10 | 上下文压缩 | LLMChainExtractor 抽取式压缩 |
| 11 | 长上下文重排 | LongContextReorder 防"迷失在中间" |

## 安装

```bash
pip install modularragagain
```

需要环境变量：

```bash
set DASHSCOPE_API_KEY=sk-xxxx        # Windows
export DASHSCOPE_API_KEY=sk-xxxx     # Linux / macOS
```

## 快速开始

```python
import os
from modularragagain import ModularRAG

rag = ModularRAG({
    "persist_directory": "./chroma_db",   # 需已存在的 Chroma 知识库
    "collection_name": "study",
    "embedding_model": "text-embedding-v4",
})

question = "什么是 RAG？"
query = rag.normalize_query(question)
query = rag.rewrite_query(query)
hyde = rag.generate_hyde(query)

docs = rag.vector_search(hyde, rag.db, top_k=5)
docs = rag.deduplicate(docs)
docs = rag.rerank_documents(query, docs, top_k=3)
docs = rag.similarity_filter(query, docs, rag.embeddings, threshold=0.4)
docs = rag.compress_context(query, docs)
docs = rag.reorder_documents(docs)

for d in docs:
    print(d.metadata.get("cosine_similarity"), d.page_content[:80])
```

完整可运行示例见 [`examples/demo.py`](examples/demo.py)，它会在本地自建一个测试知识库并跑通全部 11 个模块。

## 依赖

`langchain-core` / `langchain-openai` / `langchain-community` / `langchain-chroma` / `langchain-classic` / `chromadb` / `dashscope` / `jieba` / `python-dotenv`

## License

未指定（如需开源请补充 LICENSE）。
