Metadata-Version: 2.4
Name: mkd
Version: 0.3.0
Summary: Local Markdown document tooling for counting, transcription, and conversion
Requires-Python: >=3.11
Description-Content-Type: text/markdown
Requires-Dist: beautifulsoup4>=4.12
Requires-Dist: lxml>=5
Requires-Dist: pypdf>=5.0.0
Requires-Dist: python-docx>=1.2.0
Requires-Dist: rapidocr-onnxruntime>=1.2.3
Requires-Dist: typer>=0.21.1

# `mkd`

**Contents:**

- [What is `mkd`?](#what-is-mkd)
  - [What can `mkd` do?](#what-can-mkd-do)
  - [Why can I trust `mkd`?](#why-can-i-trust-mkd)
  - [Who is `mkd` for?](#who-is-mkd-for)
- [Getting started](#getting-started)
  - [Prerequisites](#prerequisites)
  - [Installation](#installation)
  - [Core commands](#core-commands)
  - [Testing](#testing)
  - [Documentation map](#documentation-map)
- [Other features](#other-features)
  - [Transcription cache](#transcription-cache)
  - [Word-count markers](#word-count-markers)
- [Design and safety](#design-and-safety)
  - [*Guaranteed* invariants](#guaranteed-invariants)
  - [External dependencies](#external-dependencies)

## What is `mkd`?

`mkd` is a **local-first CLI oriented around Markdown documents** that:

- gathers an essential family of Markdown chores under a ***single* executable**, run with a path-explicit command
- **holds *no* opinion about file layout** in your project
- keeps heavyweight tools ([Pandoc](https://pandoc.org), [Poppler](https://poppler.freedesktop.org), and [RapidOCR](https://github.com/RapidAI/RapidOCR)) *outside* the Python package 

### What can `mkd` do?

- **Transcribe** images and PDFs into **page-sectioned Markdown**, with a **content-addressed local cache** for large jobs
- **Convert between DOCX/HTML/Markdown** via Pandoc, with a **tweakable reference DOCX** letting you easily specify your desired house style
- **Split/join Markdown files**, backed by manifests to survive edit cycles without losing order
- **Reflow** (e.g. to one sentence per line) for diff-clean prose review
- **Word counts** (including *tags* allowing **multiple custom section-based counts at once**)
- **Proofing** via deterministic OCR cleanup & *zero* external LLM calls

  > Only *safe* fixes auto-apply; everything uncertain is surfaced for review.

### Why can I trust `mkd`?

> [!TIP]
> `mkd`'s **local-first trust model** is *intentionally boring* about trust boundaries.
> 
> In particular, **`mkd` will *never*:** 
> 
> - fetch remote URLs
> - check remote ilnks
> - log into services
> - talk to a server
> - crawl repos
> - watch directories

- The *only* automatic side effect is the **transcription cache**, which is placed at `${XDG_CACHE_HOME:-~/.cache}/mkd` by default.
- Commands read the input paths you pass and write to either:
 
  - stdout
  - explicit output paths
  - documented paths derived from those inputs

- Conversion and OCR commands *may* create and use temporary files under the system temp directory (i.e. `/tmp`), then clean them up after processing completes.

### Who is `mkd` for?

- **Who it is for**: an engineer or writer who wants these operations *boring, scriptable, and reproducible*.
- **What it is not**: a document server, a network client, or a policy engine.

## Getting started

### Prerequisites

#### Global/mandatory

Ensure the following are available (including in the `$PATH`, if applicable) before proceeding:

- Python `>= 3.11`
- [uv](https://docs.astral.sh/uv/)

#### Subcommand-specific

The following dependencies are only required by certain `mkd` subcommands:

| Dependency | Subcommands requiring it |
| --- | --- |
| [Pandoc](https://pandoc.org) | `count`, `from-docx`, `from-html`, `to-docx`, `reference` |
| [Poppler](https://poppler.freedesktop.org) | `transcribe pdf` |

> [!TIP]
> On macOS, both can be installed easily via Homebrew:
>
> ```bash
> brew install pandoc poppler
> ```

### Installation

Use the checkout directly while developing:

```bash
uv sync
uv run mkd --help
```

To **install `mkd` machine-wide** (based on a local checkout of the repo), run this in your checkout's root dir:

```bash
uv tool install -e . --force
mkd --help
```

> [!TIP]
> - Examples below use the bare `mkd`. 
> - From an **unmanaged checkout,** prefix them with `uv run`.
> - For **project-pinned invocation** (via wrapper scripts), see [docs/operations.md](docs/operations.md).

### Core commands

> [!IMPORTANT]
> **Run `mkd doctor` first** to confirm the environment before working on live documents (*especially* after installing or upgrading `mkd` or an external tool).

- Check the environment, then preflight and count a draft:

  ```bash
  mkd doctor --smoke                      # probe pandoc, Poppler, and RapidOCR
  mkd check paper.md --fail-on warning    # fail the command on warnings, not just errors
  mkd count paper.md
  ```
  
- Format prose for line-oriented review:
  
  ```bash
  mkd sentences draft.md formatted.md     # write to a new file
  mkd sentences draft.md --in-place       # rewrite the input (explicit opt-in)
  ```
  
- **Transcribe source material** into page-sectioned Markdown:
  
  ```bash
  mkd transcribe pdf reading.pdf --output reading.md
  mkd transcribe pdf reading.pdf --page-range 12 18 --mode auto --output reading-p12-p18.md
  mkd transcribe images pages/ --output pages.md
  ```
  
- **Proof OCR output**, then split it by page and join it back after editing:
  
  ```bash
  mkd proof transcript.md proofed.md --report proof-report.json
  mkd split proofed.md parts --by page --manifest
  mkd join --manifest parts/_mkd-split.json --output edited.md
  ```
  
- **Convert across document formats:**
  
  ```bash
  mkd from-docx authoritative.docx draft.md --keep-comments
  mkd from-html export.html export.md --selector '.WordSection1'
  mkd to-docx draft.md final.docx --reference-doc reference.docx
  ```

### Testing

- Run the **full test suite**:

  ```bash
  uv run pytest
  ```

  - Tests target module boundaries first, then CLI wiring.
  - `tests/test_cli.py` drives the app through Typer's `CliRunner`.
  - The test suite runs *fully offline*, matching the runtime contract.
  <p><p/>

  > See [docs/testing.md](docs/testing.md) for structure, mocks, and smoke checks.

- **Lint**:
  
  ```bash
  # checks:
  # - line length 100
  # - rule sets E, F, I, N, W
  uv run ruff check .
  ```

### Documentation map

| Document | Contains | 
| --- | --- |
| [docs/commands.md](docs/commands.md) | **Full CLI reference** covering options, outputs, & failure semantics |
| [docs/workflows.md](docs/workflows.md) | **End-to-end recipes** for transcription, proofing, checks, conversion, & split/join |
| [docs/operations.md](docs/operations.md) | Install modes, environment checks, & stdout/file behavior |
| [docs/architecture.md](docs/architecture.md) | Module boundaries, external-tool edges, cache model, & invariants |
| [docs/cache.md](docs/cache.md) | Cache location, invalidation, refresh, & cleanup |
| [docs/troubleshooting.md](docs/troubleshooting.md) | Common failures & how to narrow them |
| [docs/extension-guide.md](docs/extension-guide.md) & [docs/development.md](docs/development.md) | Adding commands & following repo conventions |


## Other features

### Transcription cache

To make large jobs cheap to rerun, `transcribe pdf` and `transcribe images` cache successful pages by default.

#### CLI flags

- Default runs reuse valid successful entries and retry misses or prior failures.
- `--refresh` ignores cached successes and rewrites them.
- `--no-cache` disables reads and writes for a single run.
- `--cache-dir PATH` uses `PATH` as the app cache root for a single run.

#### Cache location

By default, the cache lives at `${XDG_CACHE_HOME:-~/.cache}/mkd`.

```bash
mkd cache path
mkd cache prune --older-than 30d --dry-run
mkd cache clean --force
```

#### Lifecycle

- Cache entries key on:

  - the source bytes
  - the `mkd` version
  - the processing options
  - the signatures of the OCR and Poppler tools involved

- This is so that **changes in the user's choice of tools or options invalidate stale entries *automatically***. 
  
> [!TIP]
> - Cache data can contain extracted document text and diagnostic source paths.
> - Use `--no-cache` for a sensitive one-off run.
> - Full layout and invalidation rules can be found at [docs/cache.md](docs/cache.md).

### Word-count markers

- By default, `mkd count` counts the whole rendered document. 
- To count only selected regions, wrap them:

  ```markdown
  <!-- word-count-begin -->
  This counts.
  <!-- word-count-end -->
  ```

  - Short forms (`<!-- wcb -->` and `<!-- wce -->`) are also accepted.

> [!TIP]
> Run `mkd check` before trusting marked counts in automation.

## Design and safety

> The core is deliberately small and boring: every capability is a thin CLI shell over a Typer-free module that can be tested in isolation.

- **Local-first and path-based**: commands take explicit input/output paths, write local files or stdout, and perform no network I/O.
- **Thin CLI over tested modules**: `src/mkd/cli.py` owns only Typer wiring, option validation, and exit codes; domain logic lives in modules that return plain values, dataclasses, or typed exceptions.
- **Heavyweight tools at narrow edges**: Pandoc, Poppler, and RapidOCR are resolved with `shutil.which`, invoked through `subprocess`, and translated into `mkd` objects or typed errors immediately, so foreign formats never leak inward.
- **Failures are contract, not surprise**: expected problems raise one of twelve `MkdError` subclasses, which the CLI renders as `mkd <command>: <message>` rather than as a stack trace.

### *Guaranteed* invariants

**`mkd`'s only persistent side effects are documented and local:** explicit output files, explicit in-place rewrites, the app cache at `${XDG_CACHE_HOME:-~/.cache}/mkd` unless disabled or overridden, and temporary files under the system temp directory during conversion or OCR.

- **No network, ever**: no command fetches a remote URL, and `check` validates structure and DOCX hazards statically, without invoking Pandoc.
- **No silent overwrites**: `sentences` and `proof` require `--in-place` to touch the input; `split`, `join`, and `reference` refuse risky overwrites without `--force`; `from-html` refuses an identical input/output path.
- **Clean stdout**: content-producing commands keep stdout to content and send diagnostics to stderr or explicit report files, so `mkd` composes in pipelines.
- **Stable transcript headings**: every transcript section is headed `### Page <label>`; this is a cross-module contract, so `proof` preserves it, page-based `split` recognizes it, and the cache keys on the label that produced it.

`mkd doctor` returns a stable scheme: `0` (healthy), `1` (a failed check, or a warning under `--strict`), and `2` (a usage or orchestration error). Other commands exit non-zero on a handled failure.

For the full module map, external-tool contracts, and the complete invariant list, see [docs/architecture.md](docs/architecture.md).

### External dependencies

- `mkd` ships thin and defers document-format interpretation to established tools. 
- `mkd doctor` reports which are present, their versions (and, with `--smoke`, whether they actually run); 
    
  - Missing tools surface as concise, actionable CLI errors.

| Tool | Powers | Notes |
| :--- | :--- | :--- |
| Pandoc | word counts, DOCX and HTML conversion, reference DOCX | parses Markdown to JSON; owns format interpretation |
| Poppler (`pdfinfo`, `pdftotext`, `pdftoppm`) | PDF page counts, text extraction, rasterization | text-first, with raster fallback for OCR |
| RapidOCR | OCR of images and PDF fallback pages | installed into the environment *automatically* by uv |
