Metadata-Version: 2.4
Name: wsh-rag
Version: 0.1.0
Summary: Ask natural-language questions about Singapore Workplace Safety and Health PDFs, with citations.
Project-URL: Homepage, https://github.com/LogiqlyAI/wsh-rag
Project-URL: Issues, https://github.com/LogiqlyAI/wsh-rag/issues
Author-email: Wong Hong Yu <hongyu@logiqlyai.com>
License-Expression: MIT
License-File: LICENSE
Keywords: anthropic,claude,cli,pdf,rag,singapore,workplace-safety,wsh
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Information Technology
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Classifier: Topic :: Text Processing :: Indexing
Requires-Python: >=3.11
Requires-Dist: anthropic>=0.40.0
Requires-Dist: lancedb>=0.13.0
Requires-Dist: langchain-text-splitters>=0.3.0
Requires-Dist: numpy>=1.26.0
Requires-Dist: pyarrow>=16.0.0
Requires-Dist: pymupdf>=1.24.0
Requires-Dist: rich>=13.7.0
Requires-Dist: sentence-transformers>=3.0.0
Requires-Dist: tiktoken>=0.7.0
Requires-Dist: typer>=0.12.0
Provides-Extra: dev
Requires-Dist: pytest-mock>=3.12.0; extra == 'dev'
Requires-Dist: pytest>=8.0.0; extra == 'dev'
Requires-Dist: reportlab>=4.0.0; extra == 'dev'
Requires-Dist: ruff>=0.6.0; extra == 'dev'
Description-Content-Type: text/markdown

# wsh-rag

> Ask natural-language questions about Singapore Workplace Safety and Health (WSH) PDFs — locally indexed, Claude-powered, with inline citations.

[![PyPI](https://img.shields.io/pypi/v/wsh-rag.svg)](https://pypi.org/project/wsh-rag/)
[![Python](https://img.shields.io/pypi/pyversions/wsh-rag.svg)](https://pypi.org/project/wsh-rag/)
[![CI](https://github.com/LogiqlyAI/wsh-rag/actions/workflows/ci.yml/badge.svg)](https://github.com/LogiqlyAI/wsh-rag/actions/workflows/ci.yml)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)

`wsh-rag` is a small command-line tool that points at a folder of PDFs — the WSH Act, MOM Codes of Practice, bizSAFE materials, your collected Risk Assessments — builds a local vector index, and answers natural-language questions about them with **inline citations back to the source file and page**.

It's built for safety officers, employers, and bizSAFE candidates who routinely need to look up obligations under Singapore's Workplace Safety and Health regime and want answers traced back to the underlying paragraph.

## Install

```sh
pip install wsh-rag
# or, recommended for CLI tools:
pipx install wsh-rag
```

Requires Python 3.11+.

## Quickstart

```sh
export ANTHROPIC_API_KEY=sk-ant-...

wsh-rag index ./my-wsh-pdfs

wsh-rag ask "What are an employer's duties under the WSH Act?" \
  --folder ./my-wsh-pdfs
```

That's it. The first `index` run will download the embedding model (~130 MB, one-time). Subsequent indexes and asks are fast and run entirely on your machine — only the final answering step calls the Claude API.

## What you'll see

```
╭──────────────────────── ANSWER ────────────────────────╮
│ Under the Workplace Safety and Health Act, every       │
│ employer must take reasonably practicable measures to  │
│ ensure the safety and health of employees at work [F1].│
│ This includes conducting risk assessments before any   │
│ work activity that may pose a risk [F2][F3], and       │
│ maintaining records of those assessments for at least  │
│ three years [F2].                                      │
╰────────────────────────────────────────────────────────╯
        ┏━━━━━━━━━━━━━━━━ SOURCES ━━━━━━━━━━━━━━━━━━━━━┓
        ┃  Tag  │  File                       │  Page  ┃
        ┠───────┼─────────────────────────────┼────────┨
        ┃  F1   │  WSH_Act_2006.pdf           │     14 ┃
        ┃  F2   │  COP_Risk_Management.pdf    │      8 ┃
        ┃  F3   │  COP_Risk_Management.pdf    │     12 ┃
        ┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
```

Only chunks the model actually cited appear in the Sources table. The `[F1]` markers stay in the answer so you can trace each claim back to a row.

## How it works

```
PDFs ─▶ pymupdf ─▶ page-bounded chunker ─▶ bge-small embeddings
                                                    │
                                                    ▼
                                              LanceDB (file-based)
                                                    │
question ─▶ query embedding ─▶ top-k retrieval ─────┘
                                          │
                                          ▼
                            Claude Sonnet 4.6 (with prompt caching)
                                          │
                                          ▼
                                cited answer + Sources table
```

Notable choices, briefly:

- **Page-bounded chunks** — each chunk has exactly one source page, so citations stay precise.
- **Local embeddings** — `BAAI/bge-small-en-v1.5` runs on your machine, no embedding API cost.
- **Prompt caching** — the system prompt and retrieved context are cached on the Anthropic side, so re-asking against the same context within ~5 minutes runs at a heavy discount.
- **Sources show only what was cited** — if the model invented an `[F9]`, the marker stays in the answer (so you can spot it) but the table only lists the real ones.

## Configuration

| Env var | Purpose |
|---|---|
| `ANTHROPIC_API_KEY` | Required for `wsh-rag ask`. Get one at https://console.anthropic.com/. |

### `wsh-rag index <folder>`

| Flag | Default | Notes |
|---|---|---|
| `--chunk-tokens` | 800 | Tokens per chunk. |
| `--overlap` | 100 | Token overlap between adjacent chunks (within a page). |
| `--embed-model` | `BAAI/bge-small-en-v1.5` | Any sentence-transformers model. Changing this requires re-indexing. |

### `wsh-rag ask "<question>"`

| Flag | Default | Notes |
|---|---|---|
| `--folder` / `-f` | `.` | Folder containing the `.wsh-rag/` index. |
| `--top-k` / `-k` | 5 | Chunks retrieved as context. |
| `--model` | `claude-sonnet-4-6` | Anthropic model id. |
| `--max-tokens` | 1500 | Max tokens in the answer. |
| `--embed-model` | `BAAI/bge-small-en-v1.5` | Must match what you used at index time. |

## Where the index lives

`wsh-rag` writes its index to `<folder>/.wsh-rag/` — sibling to the PDFs you indexed. It's safe to delete (`rm -rf .wsh-rag`) and trivially re-buildable. Add this to your `.gitignore`:

```
.wsh-rag/
```

## Singapore WSH context

This tool is built around the kinds of documents the Singapore Ministry of Manpower (MOM) and the Workplace Safety and Health Council publish:

- The **Workplace Safety and Health Act** (and its subsidiary regulations)
- **Codes of Practice** (e.g. CP on Risk Management, CP on Working Safely at Heights)
- **bizSAFE** training materials and audit checklists
- **MOM advisories** and circulars
- Your own Risk Assessments and Safe Work Procedures

The system prompt instructs Claude to refuse to speculate when the provided context doesn't cover a question — important because WSH guidance is precise and small wording differences matter legally.

> **Disclaimer:** `wsh-rag` is not a substitute for legal advice. Always cite the source PDF, not this tool's output, when reporting or making compliance decisions.

## Limitations (v0.1.0)

- No incremental re-indexing — `wsh-rag index` always does a full rebuild.
- No interactive chat REPL — every `ask` is one-shot.
- English text only.
- Scanned PDFs need OCR (not bundled). If `wsh-rag index` warns that a PDF produced no extractable text, it's likely scanned.
- One vector backend (LanceDB), one LLM provider (Anthropic Claude).

## Roadmap

- Hash-based incremental re-indexing (only re-embed changed PDFs)
- Hybrid retrieval (BM25 + vector) for better recall on specific clause numbers
- Interactive `wsh-rag chat` REPL with multi-turn context
- Optional OCR via tesseract for scanned PDFs

## Development

```sh
git clone https://github.com/LogiqlyAI/wsh-rag
cd wsh-rag
python -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"

ruff check . && ruff format --check .
pytest
```

The integration test in `tests/test_cli.py` mocks the Anthropic client and uses a tiny reportlab-generated PDF, so the suite runs offline and without an API key.

## License

[MIT](LICENSE).

`wsh-rag` itself is MIT-licensed. Note that **PyMuPDF**, used for PDF parsing, is AGPL-licensed — `wsh-rag` only depends on it at runtime, so distributing `wsh-rag` under MIT is fine, but downstream users embedding `wsh-rag` should be aware of PyMuPDF's terms.

## Acknowledgements

- Anthropic Claude for answer synthesis and prompt caching
- BAAI's `bge-small-en-v1.5` for local retrieval embeddings
- LanceDB for the file-based vector store
- PyMuPDF for fast PDF text extraction
- The Singapore Workplace Safety and Health Council and MOM for publishing the underlying guidance documents this tool is designed for
