Metadata-Version: 2.4
Name: mrta-rag
Version: 0.1.0
Summary: A local citation-aware research and teaching assistant for technical PDFs
Project-URL: Homepage, https://github.com/sanya2025/multimodal-research-teaching-assistant
Project-URL: Repository, https://github.com/sanya2025/multimodal-research-teaching-assistant
Project-URL: Issues, https://github.com/sanya2025/multimodal-research-teaching-assistant/issues
Author: Sanja Damjanovic
License: MIT License
        
        Copyright (c) 2026 Sanja Damjanovic
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
License-File: LICENSE
Requires-Python: >=3.11
Requires-Dist: httpx>=0.27.0
Requires-Dist: jinja2>=3.1.4
Requires-Dist: numpy>=1.26.0
Requires-Dist: ollama>=0.2.0
Requires-Dist: opentelemetry-api>=1.25.0
Requires-Dist: pillow>=10.0.0
Requires-Dist: pydantic-settings>=2.2.0
Requires-Dist: pydantic>=2.6.0
Requires-Dist: python-dotenv>=1.0.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: structlog>=24.1.0
Provides-Extra: all
Requires-Dist: accelerate>=0.30.0; extra == 'all'
Requires-Dist: black>=24.4.0; extra == 'all'
Requires-Dist: build>=1.2.0; extra == 'all'
Requires-Dist: deepeval>=0.21.0; extra == 'all'
Requires-Dist: einops>=0.8.0; extra == 'all'
Requires-Dist: faiss-cpu>=1.8.0; extra == 'all'
Requires-Dist: fastapi>=0.111.0; extra == 'all'
Requires-Dist: huggingface-hub>=0.23.0; extra == 'all'
Requires-Dist: ipykernel>=6.29.0; extra == 'all'
Requires-Dist: jupyterlab>=4.2.0; extra == 'all'
Requires-Dist: langchain-community>=0.2.0; extra == 'all'
Requires-Dist: langchain-core>=0.2.0; extra == 'all'
Requires-Dist: langchain-text-splitters>=0.2.0; extra == 'all'
Requires-Dist: langchain>=0.2.0; extra == 'all'
Requires-Dist: matplotlib>=3.8.0; extra == 'all'
Requires-Dist: mypy>=1.10.0; extra == 'all'
Requires-Dist: nest-asyncio>=1.6.0; extra == 'all'
Requires-Dist: open-clip-torch>=2.24.0; extra == 'all'
Requires-Dist: opentelemetry-exporter-otlp-proto-grpc>=1.25.0; extra == 'all'
Requires-Dist: opentelemetry-sdk>=1.25.0; extra == 'all'
Requires-Dist: pandas>=2.2.0; extra == 'all'
Requires-Dist: pdfplumber>=0.11.0; extra == 'all'
Requires-Dist: pip-audit>=2.7.0; extra == 'all'
Requires-Dist: pymupdf>=1.24.0; extra == 'all'
Requires-Dist: pytesseract>=0.3.10; extra == 'all'
Requires-Dist: pytest-asyncio>=0.23.0; extra == 'all'
Requires-Dist: pytest>=8.2.0; extra == 'all'
Requires-Dist: python-multipart>=0.0.9; extra == 'all'
Requires-Dist: qdrant-client>=1.9.0; extra == 'all'
Requires-Dist: ruff>=0.4.0; extra == 'all'
Requires-Dist: sentence-transformers>=2.7.0; extra == 'all'
Requires-Dist: streamlit>=1.35.0; extra == 'all'
Requires-Dist: tiktoken>=0.7.0; extra == 'all'
Requires-Dist: torch>=2.2.0; extra == 'all'
Requires-Dist: tqdm>=4.66.0; extra == 'all'
Requires-Dist: transformers>=4.41.0; extra == 'all'
Requires-Dist: twine>=5.0.0; extra == 'all'
Requires-Dist: unstructured>=0.13.0; extra == 'all'
Requires-Dist: uvicorn[standard]>=0.30.0; extra == 'all'
Requires-Dist: watchdog>=4.0.0; extra == 'all'
Provides-Extra: api
Requires-Dist: fastapi>=0.111.0; extra == 'api'
Requires-Dist: python-multipart>=0.0.9; extra == 'api'
Requires-Dist: uvicorn[standard]>=0.30.0; extra == 'api'
Provides-Extra: dev
Requires-Dist: black>=24.4.0; extra == 'dev'
Requires-Dist: build>=1.2.0; extra == 'dev'
Requires-Dist: ipykernel>=6.29.0; extra == 'dev'
Requires-Dist: jupyterlab>=4.2.0; extra == 'dev'
Requires-Dist: matplotlib>=3.8.0; extra == 'dev'
Requires-Dist: mypy>=1.10.0; extra == 'dev'
Requires-Dist: opentelemetry-sdk>=1.25.0; extra == 'dev'
Requires-Dist: pandas>=2.2.0; extra == 'dev'
Requires-Dist: pip-audit>=2.7.0; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.23.0; extra == 'dev'
Requires-Dist: pytest>=8.2.0; extra == 'dev'
Requires-Dist: ruff>=0.4.0; extra == 'dev'
Requires-Dist: tqdm>=4.66.0; extra == 'dev'
Requires-Dist: twine>=5.0.0; extra == 'dev'
Requires-Dist: watchdog>=4.0.0; extra == 'dev'
Provides-Extra: eval
Requires-Dist: deepeval>=0.21.0; extra == 'eval'
Requires-Dist: nest-asyncio>=1.6.0; extra == 'eval'
Provides-Extra: llm
Requires-Dist: accelerate>=0.30.0; extra == 'llm'
Requires-Dist: huggingface-hub>=0.23.0; extra == 'llm'
Requires-Dist: torch>=2.2.0; extra == 'llm'
Requires-Dist: transformers>=4.41.0; extra == 'llm'
Provides-Extra: multimodal
Requires-Dist: einops>=0.8.0; extra == 'multimodal'
Requires-Dist: open-clip-torch>=2.24.0; extra == 'multimodal'
Provides-Extra: ocr
Requires-Dist: pytesseract>=0.3.10; extra == 'ocr'
Requires-Dist: unstructured>=0.13.0; extra == 'ocr'
Provides-Extra: otel
Requires-Dist: opentelemetry-exporter-otlp-proto-grpc>=1.25.0; extra == 'otel'
Requires-Dist: opentelemetry-sdk>=1.25.0; extra == 'otel'
Provides-Extra: pdf
Requires-Dist: pdfplumber>=0.11.0; extra == 'pdf'
Requires-Dist: pymupdf>=1.24.0; extra == 'pdf'
Provides-Extra: qdrant
Requires-Dist: qdrant-client>=1.9.0; extra == 'qdrant'
Provides-Extra: retrieval
Requires-Dist: faiss-cpu>=1.8.0; extra == 'retrieval'
Requires-Dist: langchain-community>=0.2.0; extra == 'retrieval'
Requires-Dist: langchain-core>=0.2.0; extra == 'retrieval'
Requires-Dist: langchain-text-splitters>=0.2.0; extra == 'retrieval'
Requires-Dist: langchain>=0.2.0; extra == 'retrieval'
Requires-Dist: sentence-transformers>=2.7.0; extra == 'retrieval'
Requires-Dist: tiktoken>=0.7.0; extra == 'retrieval'
Provides-Extra: streamlit
Requires-Dist: streamlit>=1.35.0; extra == 'streamlit'
Description-Content-Type: text/markdown

# Multimodal AI Research & Teaching Assistant

Upload a research paper PDF, index it locally, and get answers grounded in the
document with exact page citations — no cloud API required. The system runs
entirely on your machine using Ollama for language models and FAISS for vector
search. This repository is also a 10-notebook tutorial series covering every
component end-to-end, from PDF ingestion to evaluation.

## Features

- Upload a PDF and build a searchable FAISS index
- Ask questions and receive answers with page citations
- Five teaching modes: beginner, graduate, interview prep, quiz generation, and figure explanation
- Optional figure captioning with a vision-language model
- Source-scoped retrieval — Explain figure mode constrains search to the selected document
- Duplicate upload detection — re-uploading the same PDF returns a cached response instantly
- OpenTelemetry tracing — per-request spans with retrieval scores, token counts, and latency
- Fully local: Ollama + Hugging Face, no API keys required
- Production-style architecture with typed modules, API/UI separation, Docker, testing, CI, evaluation, and observability

## Prerequisites

**Required:**

- [Docker Desktop](https://www.docker.com/products/docker-desktop)
- [Ollama](https://ollama.com)
- Git

Text model:

```bash
ollama pull llama3.2:latest
```

**Optional** — enables figure and image captioning (~6 GB):

```bash
ollama pull qwen2.5vl:latest
```

The vision model is not required for text-only PDF question answering. When it
is not installed, the **Explain figure** mode falls back to text-based
retrieval and shows an in-app prompt with the install command.

## Quick start

```bash
cp .env.example .env
ollama pull llama3.2:3b
docker compose up --build
```

Optional — enable figure captioning:

```bash
ollama pull qwen2.5vl:7b
```

Open:

- UI: <http://localhost:8501>
- API docs: <http://localhost:8000/docs>

**Demo workflow:**

1. In the sidebar, upload `data/sample/attention_is_all_you_need.pdf`
2. Click **Index document**
3. Ask: *"What problem does self-attention solve?"*

## Python package

The core library is distributed as `mrta-rag` on PyPI:

```bash
# Core only (config, schemas, LLM client, prompts)
pip install mrta-rag

# Add PDF ingestion
pip install "mrta-rag[pdf]"

# Add chunking, embeddings, and FAISS vector search
pip install "mrta-rag[retrieval]"

# Full install (matches the Docker environment)
pip install "mrta-rag[all]"
```

```python
import mrta

print(mrta.__version__)   # 0.1.0

# Core API available after pip install mrta-rag:
from mrta import rag_query, LLMClient, Settings, load_prompt

# Requires mrta-rag[pdf]:
from mrta import load_pdf, chunk_pdf

# Requires mrta-rag[retrieval]:
from mrta import Embedder, VectorStore
```

## Development

### Local setup (without Docker)

```bash
python -m venv .venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate
pip install -e ".[all]"
```

Run the backend and frontend in separate terminals:

```bash
uvicorn apps.api.main:app --reload --port 8000
streamlit run apps/streamlit/app.py
```

### Environment switching

Config is loaded from `configs/{MRTA_ENV}.yaml`, with env vars and `.env`
taking priority:

```bash
MRTA_ENV=test pytest   # lighter models, fast CI
MRTA_ENV=dev pytest    # full dev config
```

### Tests

```bash
pytest
pytest tests/unit/        # unit tests only
pytest tests/evaluation/  # retrieval gate tests
```

### Observability

Tracing is controlled by three `.env` variables:

```bash
ENABLE_TRACING=true           # activate the OTEL SDK
OTEL_CONSOLE_EXPORTER=true    # print spans to stdout (local dev)
OTEL_SERVICE_NAME=mrta
OTEL_EXPORTER_OTLP_ENDPOINT=  # set to export to Jaeger / Tempo
```

With console export enabled, each `/ask` call prints a span to the API logs showing
retrieval scores, cited sources, token counts, and end-to-end latency.

### Linting and type checking

```bash
ruff check src/ tests/ apps/
black --check src/ tests/ apps/
.venv311/bin/mypy src/ apps/ --ignore-missing-imports
```

> **Note:** Use a Python 3.11 virtual environment for `mypy`. The default
> `.venv` uses Python 3.14, whose NumPy stubs use syntax that mypy rejects
> when `python_version = "3.11"` is set. CI uses Python 3.11 and passes.

### Tutorial notebooks

```bash
jupyter lab notebooks/
```

Two parallel versions of the 10-part series:

- **`notebooks/production/`** — imports from `src/mrta/`; the reference implementation
- **`notebooks/tutorials/`** — every function defined inline; use for learning

| # | Phase | Topic |
|---|-------|-------|
| 0 | Setup | Repo scaffold, Ollama, Hugging Face |
| 1 | Ingestion | PyMuPDF text and image extraction |
| 2 | Chunking | Fixed, recursive, and semantic strategies |
| 3 | Embeddings | sentence-transformers + FAISS index |
| 4 | RAG | End-to-end pipeline with citations |
| 5 | Backend | FastAPI endpoints and Pydantic schemas |
| 6 | Frontend | Streamlit upload, ask, cite |
| 7 | Multimodal | Figure extraction, CLIP, VLM captioning |
| 8 | Teaching modes | Prompt templates for different audiences |
| 9 | Evaluation | DeepEval metrics, structured logs, Docker |

### Architecture and design decisions

- Tech stack, system diagram, and repo layout: [`docs/architecture/overview.md`](docs/architecture/overview.md)
- Key design decisions (FAISS vs Qdrant, Ollama vs API, etc.): [`docs/adr/`](docs/adr/)

## Limitations

- Math is rendered as text; LaTeX-aware parsing would improve recall on equation-heavy papers.
- Table extraction is basic; ColPali or `unstructured` would help for table-heavy domains.
- Reranking is a stub; adding a cross-encoder (`bge-reranker-base`) is a one-day improvement.
- No multi-document graph reasoning yet — a clear next step toward an "agentic" research assistant.

## License

This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.
