Metadata-Version: 2.4
Name: yapstack
Version: 0.2.0
Summary: Talk to your files. Local RAG that actually works — llama.cpp + turbovec.
License-Expression: MIT
Project-URL: Source, https://github.com/shubhayu-64/yapcache
Keywords: rag,llm,retrieval,yapstack,llamacpp
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: click>=8.1
Requires-Dist: lancedb>=0.12
Requires-Dist: rank-bm25>=0.2
Requires-Dist: pyarrow>=14
Requires-Dist: huggingface-hub>=0.20
Requires-Dist: pydantic>=2
Requires-Dist: pymupdf>=1.24
Requires-Dist: pillow>=10
Requires-Dist: python-docx>=1.1
Requires-Dist: pandas>=2
Requires-Dist: openpyxl>=3.1
Requires-Dist: numpy>=1.26
Provides-Extra: metal
Requires-Dist: turbovec>=0.3; extra == "metal"
Requires-Dist: mlx>=0.14; extra == "metal"
Provides-Extra: embed
Requires-Dist: sentence-transformers>=3; extra == "embed"
Requires-Dist: keybert>=0.8; extra == "embed"
Provides-Extra: llm
Requires-Dist: llama-cpp-python>=0.3; extra == "llm"
Provides-Extra: all
Requires-Dist: yapstack[embed,llm,metal]; extra == "all"
Dynamic: license-file

# YapCache

**Your files been yapping? Finally, you can yap back.**

YapCache is a local RAG pipeline that indexes your docs — PDFs, spreadsheets,
meeting notes, images, voice recordings — and lets you ask stuff in plain English.
No data leaves your machine. No API keys. No wifi needed. Pure ✨ local vibes ✨.

```
yap ingest ./docs
yap ask "What did the quarterly report say about revenue?"
yap ask "Summarise the investor deck" --stream
yap interactive
```

## Quick start

```bash
# 1. Install
pip install yapstack[all]

# 2. Download a model (default: Llama 3.2 3B Q4)
huggingface-cli download bartowski/Llama-3.2-3B-Instruct-GGUF \
  Llama-3.2-3B-Instruct-Q4_K_M.gguf --local-dir ./models

# 3. Point it at your documents
cp ~/Documents/report.pdf ./data/

# 4. Ingest
yap ingest ./data

# 5. Ask
yap ask "What are the key findings?"
yap ask "Summarise the financials" --stream
yap interactive
```

### One-shot setup script

```bash
bash setup.sh              # creates .venv, installs deps, downloads model
source .venv/bin/activate
yap ingest ./data
yap ask "your question"
```

## Features

| | |
|---|---|
| **Multi-format parsing** | PDF, DOCX, XLSX, CSV, images (EXIF + optional LLM captions), audio (Whisper), Markdown, HTML, code, plain text |
| **Hybrid search** | Semantic (cosine ANN via LanceDB) + keyword (BM25) fused with Reciprocal Rank Fusion |
| **Query optimisation** | HyDE (hypothetical document embedding), query rewriting, sub-question expansion |
| **Cross-encoder reranking** | `ms-marco-MiniLM-L-6-v2` scores candidate chunks against the query |
| **Streaming** | Token-by-token output, interactive REPL mode |
| **Deduplication** | Re-running ingest only adds new content (MD5-based) |
| **Sources** | Every answer cites which files the information came from |
| **Evaluation** | NDCG, MRR, Recall, Precision for retrieval; BLEU, ROUGE-L, faithfulness, latency for answers |
| **GPU auto-detect** | Metal on macOS, CUDA on Linux, CPU fallback — no config needed |
| **Cross-platform** | macOS (Apple Silicon), Linux (NVIDIA), Windows (CPU) |

## CLI reference

```
yap [OPTIONS] COMMAND [ARGS]...

Commands:
  ingest       Parse, chunk, embed, index
  ask          Answer a question
  interactive  REPL session
  evaluate     Run eval harness
  status       Show index stats
```

### `yap ingest`

```
yap ingest [DATA_DIR] [options]
```

| Option | Default | Description |
|---|---|---|
| `DATA_DIR` | `./data` | Directory to ingest (recursive) |
| `--index-dir` | `./index_store` | Where to write the index |
| `--model` | config default | HF model spec (for image captioning) |
| `--caption` | off | Enable LLM image captioning (loads model, slower) |

Ingest walks the directory, parses every supported file, splits text into sentence-boundary chunks, extracts keywords (KeyBERT), embeds via turbovec/sentence-transformers, writes to LanceDB + BM25. Deduplicates by MD5 hash — safe to re-run.

### `yap ask`

```
yap ask "your question" [options]
```

| Option | Default | Description |
|---|---|---|
| `--index-dir` | `./index_store` | Index to query |
| `--model` | config default | HF `repo:file.gguf` or local path to GGUF |
| `--no-optimize` | off | Skip HyDE / rewriting (faster, less accurate) |
| `--no-rerank` | off | Skip cross-encoder reranking (faster) |
| `--stream` | off | Stream output tokens as they're generated |
| `--show-sources` / `--no-show-sources` | on | Print source file paths |

### `yap interactive`

```
yap interactive [--index-dir PATH] [--model PATH]
```

Starts a REPL. Type `exit` or Ctrl-C to quit. Streaming is always on.

### `yap evaluate`

```
yap evaluate [--dataset PATH] [--index-dir PATH] [--model PATH]
```

Runs retrieval and answer quality metrics against a JSONL dataset. See [eval format](#eval-format) below.

### `yap status`

```
yap status [--index-dir PATH]
```

Prints chunk count, BM25 document count, embedding dimension, and model path.

## Install

### pip

```bash
pip install yapstack[all]
```

Platform extras (auto-detected at runtime, no flag needed):

| Extra | Installs | Benefit |
|---|---|---|
| `[all]` | Everything | Recommended |
| `[metal]` | `turbovec` + `mlx` | 3-5x faster embeddings on Apple Silicon |
| `[embed]` | `sentence-transformers` + `keybert` | Keyword extraction + reranking |
| `[llm]` | `llama-cpp-python` | LLM inference |

### GPU acceleration (llama.cpp build)

Default `llama-cpp-python` installs with CPU support. For GPU:

```bash
# macOS Metal
CMAKE_ARGS="-DLLAMA_METAL=on" pip install llama-cpp-python --force-reinstall

# Linux CUDA
CMAKE_ARGS="-DLLAMA_CUDA=on" pip install llama-cpp-python --force-reinstall
```

The CLI auto-detects your GPU and sets `n_gpu_layers` accordingly — no code changes needed.

## Models

Default: `bartowski/Llama-3.2-3B-Instruct-GGUF` (Q4_K_M, ~2 GB).

Override with any GGUF from HuggingFace:

```bash
yap ask "question" --model "author/model:file.q4_k_m.gguf"
yap ask "question" --model /path/to/local/model.gguf
```

The model is downloaded to `~/.cache/yapstack/models/` on first use.

## Configuration

All settings live in `YapConfig` (pydantic). Override fields programmatically:

```python
from yapstack.config import YapConfig

cfg = YapConfig(
    model="author/model:file.gguf",
    chunk_size=384,
    top_k_rerank=10,
    temperature=0.3,
)
```

| Field | Default | Description |
|---|---|---|
| `model` | `bartowski/...:...Q4_K_M.gguf` | HF repo:file or local path |
| `chunk_size` | 512 | Words per chunk |
| `chunk_overlap` | 64 | Overlap words between chunks |
| `embed_model` | `nomic-ai/nomic-embed-text-v1.5` | Sentence transformer for embeddings |
| `embed_dim` | 768 | Embedding dimension |
| `matryoshka_dim` | `None` | Set `256` for 3x smaller index |
| `top_k_semantic` | 20 | Raw semantic candidates |
| `top_k_bm25` | 20 | Raw keyword candidates |
| `top_k_rerank` | 5 | Final reranked results |
| `rrf_k` | 60 | RRF smoothing constant |
| `n_ctx` | 4096 | LLM context window |
| `temperature` | 0.1 | Generation temperature |
| `max_tokens` | 1024 | Max answer length |
| `context_token_budget` | 2048 | Tokens reserved for retrieved context |

## Pipeline overview

```
Data folder → Parsers (PDF, DOCX, images, audio, tables, text)
                → Sentence chunker (512w, 64w overlap)
                    → Keyword extractor (KeyBERT)
                        → Embedder (turbovec / sentence-transformers)
                            → LanceDB vector store + BM25 index

At query time:
  Query → HyDE + Rewrite + Expand
            → Semantic search + BM25 search
                → RRF fusion
                    → Cross-encoder reranker
                        → Context builder (token budget)
                            → LLM (llama.cpp) → Answer + sources
```

## Eval format

Create `eval/dataset.jsonl`:

```jsonl
{"query": "What is the main topic?", "relevant_ids": ["uuid-1", "uuid-2"], "reference": "The main topic is..."}
{"query": "Who authored the report?", "relevant_ids": ["uuid-3"], "reference": "The report was authored by..."}
```

```bash
yap evaluate
```

Output:

```
── Retrieval Metrics ──────────────────────
  ndcg@5               0.7842
  mrr                  0.8100
  recall@5             0.7200

── Answer Metrics ──────────────────────────
  bleu (mean)          0.3214
  rouge-l f1 (mean)    0.5631
  faithfulness (mean)  0.8900
  latency p50          3.21s
```

## Project layout

```
yapstack/
├── pyproject.toml           # Build config + deps
├── setup.sh                 # One-shot setup script
├── README.md
│
├── src/yapstack/
│   ├── cli.py               # Click CLI (ingest / ask / interactive / evaluate / status)
│   ├── config.py             # Pydantic YapConfig
│   ├── _model.py             # HF model download + GPU auto-detect
│   │
│   ├── ingest/
│   │   ├── pipeline.py       # Orchestrator
│   │   ├── loader.py         # File discovery + parser routing
│   │   ├── chunker.py        # Sentence-boundary chunker
│   │   ├── metadata.py       # KeyBERT keyword extraction
│   │   └── parsers/          # PDF, DOCX, image, table, text, audio parsers
│   │
│   ├── index/
│   │   ├── embedder.py       # Auto-detect: turbovec → sentence-transformers
│   │   ├── vector_store.py   # LanceDB wrapper (HNSW cosine)
│   │   └── bm25_store.py     # rank-bm25 wrapper (persistent)
│   │
│   ├── retrieve/
│   │   ├── searcher.py       # Hybrid search orchestrator
│   │   ├── query_optimizer.py# HyDE + rewrite + sub-query expansion
│   │   ├── fusion.py         # RRF + cross-encoder reranker
│   │   └── fallback.py       # Low-confidence / no-result handling
│   │
│   ├── generate/
│   │   ├── llm.py            # llama-cpp-python wrapper (singleton, streaming)
│   │   ├── context_builder.py# Token-budget context packing
│   │   └── prompt_templates.py
│   │
│   └── eval/
│       ├── harness.py        # Full eval runner
│       ├── retrieval_eval.py # NDCG, MRR, Recall, Precision
│       └── answer_eval.py    # BLEU, ROUGE-L, faithfulness, latency
│
├── data/                     # Drop your documents here
├── index_store/              # LanceDB + BM25 index (auto-created)
├── models/                   # Downloaded GGUF files
└── eval/
    └── dataset.jsonl         # Your eval cases
```

## Requirements

- **Python** >= 3.10
- **macOS**: Apple Silicon (M1–M4), macOS 13+
- **Linux**: NVIDIA GPU (CUDA) or CPU
- **Windows**: CPU only
- **Disk**: ~2 GB (model) + index size
- **RAM**: ~4 GB free at inference time
