Metadata-Version: 2.4
Name: watson-lite
Version: 0.1.0
Summary: Extractive QA pipeline — no LLM, no training. BM25 + FAISS + Wikidata + cross-encoder re-ranking + roberta-base-squad2.
Project-URL: Homepage, https://github.com/daedalus/watson-lite
Project-URL: Repository, https://github.com/daedalus/watson-lite
Project-URL: Issues, https://github.com/daedalus/watson-lite/issues
Author-email: Dario Clavijo <clavijodario@gmail.com>
License: MIT
License-File: LICENSE
Requires-Python: >=3.11
Requires-Dist: bm25s>=0.1.0
Requires-Dist: faiss-cpu>=1.7.4
Requires-Dist: numpy>=1.26.0
Requires-Dist: requests>=2.31.0
Requires-Dist: scipy>=1.13.0
Requires-Dist: sentence-transformers>=2.7.0
Requires-Dist: spacy>=3.7.0
Requires-Dist: sparqlwrapper>=2.0.0
Requires-Dist: torch>=2.0.0
Requires-Dist: tqdm>=4.66.0
Requires-Dist: transformers>=4.40.0
Requires-Dist: wikipedia-api>=0.6.0
Provides-Extra: all
Requires-Dist: hatch; extra == 'all'
Requires-Dist: prospector[with-mypy,with-ruff]; extra == 'all'
Requires-Dist: pytest; extra == 'all'
Requires-Dist: pytest-cov; extra == 'all'
Requires-Dist: ruff; extra == 'all'
Provides-Extra: dev
Requires-Dist: hatch; extra == 'dev'
Requires-Dist: ruff; extra == 'dev'
Provides-Extra: lint
Requires-Dist: prospector[with-mypy,with-ruff]; extra == 'lint'
Provides-Extra: test
Requires-Dist: pytest; extra == 'test'
Requires-Dist: pytest-cov; extra == 'test'
Description-Content-Type: text/markdown

# watson-lite

A Watson-inspired extractive QA system that runs on a laptop.  
**No LLM. No trained weights of your own. No paid APIs.**

[![Python](https://img.shields.io/pypi/pyversions/watson-lite.svg)](https://pypi.org/project/watson-lite/)
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/master/assets/badge/v2.json)](https://github.com/astral-sh/ruff)

## Install

```bash
pip install watson-lite
python -m spacy download en_core_web_sm
```

## Usage

### CLI

```bash
# Single question
watson-lite "Who designed the Eiffel Tower?"
watson-lite "Who was the 44th president of the United States?"

# Interactive mode
watson-lite
```

### Python

```python
from watson_lite import WatsonLite

watson = WatsonLite()
answer = watson.answer("Who designed the Eiffel Tower?")

print(answer.answer)        # "Gustave Eiffel"
print(answer.confidence)    # 0.847
print(answer.source)        # "Eiffel Tower"
```

### Example output

```
$ watson-lite "Who was the 44th president of the United States?"

  ANSWER:     Barack Hussein Obama
  CONFIDENCE: 43.6%
  SOURCE:     Barack Obama
  URL:        https://en.wikipedia.org/wiki/Barack Obama

  Confidence breakdown:
    extraction_model: 0.592
    span_agreement: 0.2
    graph_corroboration: 0.0
    passage_rank_signal: 1.0

  Time: 44.60s
```

## API

- **`WatsonLite`** — Main orchestrator. `answer(question)` runs the full 6-stage pipeline.
- **`NLPProcessor`** — spaCy-based question classification, NER, decomposition.
- **`BM25Retriever`** — BM25 retrieval over Wikipedia REST API.
- **`VectorRetriever`** — Dense vector retrieval (sentence-transformers + FAISS).
- **`WikidataGraph`** — Structured fact enrichment from Wikidata.
- **`Ranker`** — RRF fusion + cross-encoder re-ranking.
- **`ExtractiveReader`** — Span extraction via roberta-base-squad2.
- **`ConfidenceScorer`** — Multi-signal confidence scoring.
- **`Cache`** — SQLite3 cache for Wikipedia and Wikidata responses.

## Development

```bash
git clone https://github.com/daedalus/watson-lite.git
cd watson_lite
pip install -e ".[test]"

# run tests
pytest

# format
ruff format src/ tests/

# lint + type check
prospector --with-tool ruff --with-tool mypy src/

# find unused code
vulture --min-confidence 90 src/
```

## Architecture

```
User Question → NLP (spaCy) → Decomposition → Entity Extraction
  → Parallel Retrieval (BM25 + FAISS) → Graph (Wikidata)
  → RRF Fusion → Cross-Encoder Rerank → Span Extraction → Confidence Score
```

## Models Used (all pretrained, inference only)

| Model | Purpose | Size |
|---|---|---|
| `en_core_web_sm` | spaCy NLP | ~12MB |
| `all-MiniLM-L6-v2` | Passage embeddings | ~90MB |
| `ms-marco-MiniLM-L-6-v2` | Cross-encoder reranking | ~90MB |
| `deepset/roberta-base-squad2` | Extractive span QA | ~480MB |

Total: ~670MB — runs CPU-only.

## Data Sources

- **Wikipedia REST API** — Live article retrieval
- **Wikidata REST API** — Structured entity facts (no SPARQL)

## Extending

- **Add a domain corpus**: Replace `fetch_wikipedia_passages()` with your own document loader.
- **Add more graph sources**: Wikidata REST API pattern is reusable.
- **Offline mode**: Download Wikipedia dumps and index locally with BM25 + FAISS.
