Metadata-Version: 2.4
Name: owistscribe
Version: 1.0.0
Summary: Fast, low-memory, offline OCR — CLI, local web app, and Python API in one pip install. Reads images, PDFs, and DOCX; reconstructs reading order and tables as Markdown.
Author: Openwist
License-Expression: Apache-2.0
Project-URL: Homepage, https://github.com/openwist/owistscribe
Project-URL: Repository, https://github.com/openwist/owistscribe
Project-URL: Changelog, https://github.com/openwist/owistscribe/blob/main/owistscribe/_vendor/rapidocr/CHANGES.md
Keywords: ocr,text-extraction,pdf,docx,markdown,onnxruntime,document-processing,text-recognition,offline-ocr
Platform: Any
Classifier: Development Status :: 5 - Production/Stable
Classifier: Environment :: Console
Classifier: Environment :: Web Environment
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Multimedia :: Graphics :: Capture :: Scanners
Classifier: Topic :: Scientific/Engineering :: Image Recognition
Classifier: Topic :: Text Processing
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: <4,>=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: NOTICE
Requires-Dist: pymupdf>=1.24.0
Requires-Dist: python-docx>=1.1.0
Requires-Dist: fastapi>=0.110.0
Requires-Dist: uvicorn[standard]>=0.29.0
Requires-Dist: python-multipart>=0.0.9
Requires-Dist: colorama>=0.4.6
Requires-Dist: onnxruntime>=1.18.0
Requires-Dist: pyclipper>=1.2.0
Requires-Dist: opencv_python>=4.5.1.48
Requires-Dist: numpy<3.0.0,>=1.19.5
Requires-Dist: six>=1.15.0
Requires-Dist: Shapely!=2.0.4,>=1.7.1
Requires-Dist: PyYAML
Requires-Dist: Pillow
Requires-Dist: tqdm
Requires-Dist: omegaconf!=2.2.1
Requires-Dist: requests
Requires-Dist: colorlog
Provides-Extra: desktop
Requires-Dist: tkinterdnd2; extra == "desktop"
Dynamic: license-file

# OwistScribe

**Openwist AI — building the future, intelligently.**

Fast, low-memory, **offline** OCR and document reading — one `pip install owistscribe` gives you
a CLI, a local web app, and a Python API, all sharing one engine. Built on a patched fork of
[RapidOCR](https://github.com/RapidAI/RapidOCR) (ONNX Runtime, bundled PP-OCRv6 "small"
weights) — constructing the engine makes zero network calls, ever, because the model weights
ship inside the package itself.

```bash
pip install owistscribe
owistscribe invoice.pdf
```

## Why

- **Actually offline.** No API key, no per-page cost, no network call at inference time. The
  `.onnx` model weights are bundled in the wheel.
- **Reads more than images.** PDFs and DOCX files with a real embedded text layer are read
  directly — faster and more accurate than rasterizing and OCR'ing a picture of the text. OCR
  only runs on pages that are genuinely scanned with no text layer.
- **Output is Markdown, not a text blob.** Headings and tables are reconstructed as real
  Markdown (`#` headings, `| table | cells |`) from the OCR geometry — see
  [How extraction works](#how-extraction-works) below for what that does and doesn't cover.
- **Streams.** A multi-page PDF streams page-by-page; a single image streams region-by-region
  as text is recognized, instead of a silent wait on a large file.
- **One engine, three front ends.** The CLI, the web app, and the Python API all call the same
  `owistscribe.RapidOcrEngine` — a fix or a tuning change benefits all three at once.

## Install

```bash
pip install owistscribe
```

Requires Python 3.9+. That's it — no second package, no sibling checkout, no post-install
download step. (The drag-and-drop desktop app needs one more dependency: `pip install
"owistscribe[desktop]"`.)

## Quickstart

### CLI

```bash
owistscribe invoice.png                    # print the extracted text (Markdown)
owistscribe report.pdf --json              # structured JSON instead
owistscribe scan.jpg --min-confidence 0.8  # drop low-confidence text
owistscribe web                            # launch the web app at http://127.0.0.1:8000
owistscribe desktop                        # launch the drag-and-drop desktop app
owistscribe --help                         # every command, with examples
```

Colored terminal output (headings/tables lightly highlighted) is on automatically for an
interactive terminal, off automatically when piped or redirected, and always off with
`--no-color` or the `NO_COLOR` environment variable.

### Web app

```bash
owistscribe web
```

Opens `http://127.0.0.1:8000` in your browser. Drag in an image, PDF, or DOCX; results stream
in as they're ready and render as formatted Markdown (headings, tables) with a toggle to see
the raw text. `owistscribe web --port 9000` to use a different port.

### Python API

The shape meant for pairing with a chatbot, agent, or any app that needs to read an uploaded
file before responding:

```python
from owistscribe import Scribe

owistscribe = Scribe()                      # loads the model once — build this at startup, reuse it
result = owistscribe.read("invoice.pdf")

result.text        # the full extracted text (reading-order corrected, Markdown-formatted)
result.pages        # per-page breakdown: [PageResult(page_number=1, text=..., source=...), ...]
result.headers       # detected repeated header lines, if any
result.footers       # detected repeated footer lines, if any
```

Progressive results, for a large file or a chat UI that wants to show progress:

```python
for event in owistscribe.stream("report.pdf"):
    if event["type"] == "page":
        print(f"page {event['page_number']} ready: {len(event['text'])} chars")
    elif event["type"] == "done":
        print(f"finished in {event['took_seconds']}s")
```

One-shot convenience, if you don't want to manage a `Scribe` instance yourself:

```python
from owistscribe import read
text = read("invoice.pdf").text
```

## Supported files

Images (`.png` `.jpg` `.jpeg` `.bmp` `.tiff` `.webp`), `.pdf`, `.docx`.

## How extraction works

**Reading order & tables.** RapidOCR's detection/recognition models find and read individual
text boxes — they have no concept of "this is a table" or "read top-to-bottom." That's handled
separately, in `layout.py`, using plain geometry on the returned box coordinates: cluster
regions into rows by vertical-center overlap, segment by cell count so one table's structure
can't bleed into an unrelated table on the same page, detect a shared column structure, and
render it as a Markdown table when one is found — plain reading order otherwise.

**Known, tested limitation**: a genuine two-column *page layout* (a newsletter, a two-column
résumé) can be geometrically indistinguishable from an actual two-column *table* — confirmed
with an adversarial test where two unrelated columns of ordinary prose were detected as a table
and their sentences interleaved row-by-row. Telling the two apart needs a semantic signal
(whether the left and right cell's text are topically related) this module deliberately
doesn't have. A single column of text, or a table with genuinely data-shaped cells, is
unaffected.

**Headings.** Embedded PDF text carries real font metadata (size, bold) — a bold, larger-than-
body line becomes a `#`/`##`/`###` heading. OCR'd text has no font metadata to use, so it isn't
guessed at; OCR'd pages get reading-order/table structure only, not heading markup.

**Streaming.** A multi-page PDF/DOCX streams one event per page as each is ready. A single
image streams per detected text region as recognition completes — in arrival order (grouped by
aspect ratio for recognition accuracy, not top-to-bottom), with a final corrected, reading-order
result once every region is in. See `owistscribe.Scribe.stream()`'s docstring for the exact event
shapes — the web app and the Python API use the same ones.

## Speed and accuracy — what was tested, and what changed

**The detection resolution cap was costing real accuracy — raised.** Profiling a real tall test
image (1640×5666) against upstream RapidOCR's 2000px `max_side_len` cap showed confirmed-wrong
text from small label text shrunk past legibility by the forced downscale — not missed
detections, wrong characters. Measured at four resolutions:

| `max_side_len` | avg time | regions found |
|---|---|---|
| 2000 (upstream default) | 4.26s | 89 |
| **3000 (this project's default)** | **4.64s (+9%)** | **94** |
| 4000 | 6.45s (+51%) | 98 |
| 5666 (full res, no cap) | 10.00s (+135%) | 101 |

3000 recovers effectively all of the real accuracy fix for a modest +9% time cost; 4000/5666
mostly add a few marginal tokens at a much steeper cost. Overridable per-instance:
`Scribe(max_side_len=...)` / `RapidOcrEngine(max_side_len=...)`, or via the
`OWISTSCRIBE_MAX_SIDE_LEN` environment variable for the web app.

**Tested and genuinely didn't help, said plainly:**
- Forcing onnxruntime's thread count (either `1` or `os.cpu_count()`) — no measured win over
  its own `-1`/auto-detect default; `intra_op_num_threads=1` specifically nearly *doubled*
  inference time, real proof multi-threading matters, just not worth overriding further.
- `enable_cpu_mem_arena` — a ~1.7% difference, indistinguishable from run-to-run noise.
- Multi-page parallelism (thread pool, process pool) on 2-core test hardware — no measured gain;
  may look different on a many-core machine, but that's not what was tested.
- Content-cropping before detection — tested as a speed idea, shipped as an *accuracy* fix
  instead (`preprocess.py`'s `crop_to_content`), because that's what a real padded-canvas test
  actually showed: unchanged speed, but fewer recognition errors.

Full, dated, itemized record of every patch to the vendored engine (including two real bugs
found and fixed during development — a streaming-order bug and a crash on textless images):
`owistscribe/_vendor/rapidocr/CHANGES.md`.

## Development

```bash
git clone https://github.com/openwist/owistscribe   # replace with this repo's actual URL
cd owistscribe
python -m venv venv && source venv/bin/activate
pip install -e ".[desktop]"
owistscribe web
```

## License

Apache License 2.0 — see `LICENSE`. This package vendors a fork of RapidOCR
(`owistscribe/_vendor/rapidocr/`, Apache-2.0, Copyright RapidAI) and, for the web app's
Markdown rendering, a copy of [marked](https://github.com/markedjs/marked)
(`owistscribe/static/marked.min.js`, MIT, Copyright MarkedJS/Christopher Jeffrey). See `NOTICE`
for the full attribution required by Apache-2.0 Section 4.

## What's verified and what isn't

Verified by running real files through real code and checking the actual output, not assumed:
CLI output for images/PDFs/DOCX, `--json` output shape, the web app's streaming and
non-streaming paths for every supported file type, the Markdown rendering in the browser, a
fresh `pip install` of the built wheel into a clean virtual environment (not the dev tree).

Not verified: a real build of the optional desktop app's packaged executable (PyInstaller
doesn't cross-compile, and this was built in a Linux-only sandbox) — said plainly rather than
implied. The desktop app itself runs fine from source (`owistscribe desktop`); only the "package it
into a standalone .exe/.app" step is unverified.
