# ParseDoc

> Document → AI → Markdown. A local-first CLI that converts documents into clean, structured, AI-ready Markdown.

ParseDoc turns PDFs, Word docs, PowerPoint decks, HTML pages, plain text, and images into well-structured Markdown. Libraries extract the raw facts, an optional AI model understands the structure, and a deterministic renderer produces the final output. It works **fully offline** (local rule-based structuring) and can optionally use an AI provider for smarter structuring.

---

## Features

- **Local-first**: conversion works with no AI provider configured (deterministic fallback).
- **Multi-format input**: PDF, DOCX, PPTX, HTML, TXT/MD, and images (PNG/JPG) via OCR.
- **AI structuring** (optional): `local`, `ai`, or `hybrid` modes.
- **Pluggable AI providers**: OpenAI-compatible (vLLM/LM Studio), OpenAI, Google Gemini, and Ollama (native, no SDK needed).
- **Image extraction**: pull embedded images from DOCX into an assets folder.
- **OCR**: Tesseract-backed text extraction for scanned PDFs and images.
- **Multiple output formats**: `markdown` (default), `json`, `html`, `text`.

---

## Installation

```bash
git clone <repo-url>
cd parsedoc
pip install -e .
```

This installs the `parsedoc` command. (Requires Python 3.10+.)

### Optional dependencies

| Capability | What you need |
|------------|---------------|
| PDF parsing | `pymupdf` (fitz) |
| DOCX / PPTX | `python-docx`, `python-pptx` |
| HTML | `beautifulsoup4`, `lxml` |
| OCR | [Tesseract](https://github.com/tesseract-ocr/tesseract) installed + `pytesseract` |

---

## CLI Usage

The CLI is built with Typer. Run `parsedoc --help` for the full list.

### Commands

| Command | Purpose |
|---------|---------|
| `parsedoc convert` | Convert a single document to Markdown. |
| `parsedoc batch` | Convert every matching file in a directory. |
| `parsedoc inspect` | Inspect a document's extracted structure (JSON or summary). |
| `parsedoc config` | View or edit the TOML configuration (`list`, `set`, `reset`). |
| `parsedoc version` | Print version + banner. |

### `convert`

```bash
parsedoc convert INPUT_FILE \
  --output out.md \
  --format markdown \
  --mode hybrid \
  --ai-provider ollama \
  --model qwen3 \
  --extract-images
```

**Options**

| Flag | Default | Description |
|------|---------|-------------|
| `--output`, `-o` | stdout | Write output to a file. |
| `--format`, `-f` | `markdown` | `markdown`, `json`, `html`, `text`. |
| `--mode`, `-m` | `hybrid` | `local`, `ai`, or `hybrid`. |
| `--ai-provider` | config | Override the AI provider. |
| `--model` | config | Override the model name. |
| `--temperature` | `0.2` | AI sampling temperature. |
| `--max-tokens` | `2048` | Max AI tokens. |
| `--ocr` | off | Force OCR processing. |
| `--extract-images` | off | Extract embedded images (DOCX) into `<stem>_assets/`. |
| `--quiet` / `--verbose` | off | Logging verbosity. |

### `batch`

```bash
parsedoc batch ./docs --pattern "*.docx" --format markdown --extract-images
```

### `inspect`

```bash
parsedoc inspect INPUT_FILE --format json     # full extracted structure
parsedoc inspect INPUT_FILE --format summary  # blocks / title / format
```

---

## Configuration

ParseDoc stores config as TOML (default location: `~/.config/parsedoc/config.toml`, or via `default_config_path()`). You can edit it directly or use the CLI.

```bash
parsedoc config list
parsedoc config set --key ai_provider --value ollama
parsedoc config set --key model --value qwen3
parsedoc config reset
```

### Environment variables

These override the TOML/config values at runtime:

| Variable | Maps to |
|----------|---------|
| `PARSEDOC_AI_PROVIDER` | `ai_provider` |
| `PARSEDOC_BASE_URL` | `base_url` |
| `PARSEDOC_MODEL` | `model` |
| `PARSEDOC_API_KEY` | `api_key` |
| `PARSEDOC_OCR_LANGUAGE` | `ocr_language` |

---

## Integrating AI Providers

ParseDoc supports four provider types, selected via `--ai-provider` (CLI), the `ai.provider` config key, or `PARSEDOC_AI_PROVIDER`.

| Provider value | Use case |
|----------------|----------|
| `openai-compatible` | Any OpenAI-compatible endpoint (vLLM, LM Studio, local servers, OpenRouter, etc.) |
| `openai` | OpenAI's hosted API |
| `gemini` | Google Gemini API |
| `ollama` | Ollama running locally (native HTTP, no SDK required) |

Provider is chosen by the factory in `parsedoc/ai/base.py:build_provider`.

### 1. OpenAI-compatible (default)

This is the default and works with most self-hosted / drop-in OpenAI servers.

```bash
export PARSEDOC_BASE_URL="http://localhost:11434/v1"   # e.g. Ollama's OpenAI shim
export PARSEDOC_API_KEY="local"                         # or your real key
export PARSEDOC_MODEL="qwen3"
parsedoc config set --key ai_provider --value openai-compatible
```

TOML equivalent (`~/.config/parsedoc/config.toml`):

```toml
[ai]
enabled = true
provider = "openai-compatible"
base_url = "http://localhost:11434/v1"
model = "qwen3"
api_key = "local"
temperature = 0.2
max_tokens = 4096
```

### 2. OpenAI (hosted)

```bash
export PARSEDOC_BASE_URL="https://api.openai.com/v1"
export PARSEDOC_API_KEY="sk-..."
export PARSEDOC_MODEL="gpt-4o-mini"
parsedoc config set --key ai_provider --value openai
```

### 3. Google Gemini

```bash
export PARSEDOC_API_KEY="AIza..."
export PARSEDOC_MODEL="gemini-1.5-flash"
parsedoc config set --key ai_provider --value gemini
```

> Note: the Gemini provider uses its own endpoint; `base_url` is optional and falls back to the Google Generative Language API.

### 4. Ollama (native)

No Python SDK required — ParseDoc talks to Ollama over HTTP using the standard library.

```bash
export PARSEDOC_BASE_URL="http://localhost:11434"   # Ollama root, not the /v1 shim
export PARSEDOC_MODEL="qwen3"
parsedoc config set --key ai_provider --value ollama
```

Then make sure the model is pulled:

```bash
ollama pull qwen3
```

---

## Processing Modes

| Mode | Behavior |
|------|----------|
| `local` | Pure rule-based structuring. No network calls. Fast and private. |
| `ai` | Structures content using the configured AI provider. |
| `hybrid` | Uses AI when available, falls back to local structuring on failure. |

Set via `--mode` on `convert`/`batch`, or `output`/`ai` config sections (the `provider`/`enabled` keys).

---

## Supported Input Formats

| Format | Extensions |
|--------|-----------|
| PDF | `.pdf` |
| Word | `.docx` |
| PowerPoint | `.pptx` |
| HTML | `.html`, `.htm` |
| Text / Markdown | `.txt`, `.md`, `.markdown` |
| Images | `.png`, `.jpg`, `.jpeg` (OCR) |

Check support before converting:

```bash
python -c "from parsedoc.core.detection import is_supported; print(is_supported('file.docx'))"
```

---

## Using ParseDoc as a Library

```python
from parsedoc.core.config import Config
from parsedoc.core.pipeline import Pipeline

config = Config().load_from_file()
pipeline = Pipeline(config)

markdown = pipeline.process(
    "report.docx",
    output_format="markdown",
    mode="hybrid",
    extract_images=True,
    output_path="report.md",
)
```

---

## License

See repository for license details.
