Metadata-Version: 2.5
Name: ocr-foundry
Version: 0.2.0
Summary: OCR Foundry - synthetic OCR training data with Khmer/English and opt-in multilingual rendering
Project-URL: Homepage, https://github.com/0x1d1e/ocr-foundry
Project-URL: Issues, https://github.com/0x1d1e/ocr-foundry/issues
Project-URL: Changelog, https://github.com/0x1d1e/ocr-foundry/blob/main/CHANGELOG.md
Author: Kosal Seng
License-Expression: MIT
License-File: LICENSE
Keywords: data-generation,image-generation,khmer,ocr,synthetic-data,text-recognition
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Image Recognition
Requires-Python: >=3.12
Requires-Dist: albumentations>=1.3.0
Requires-Dist: fonttools>=4.0.0
Requires-Dist: khmernormalizer>=0.0.4
Requires-Dist: lmdb>=1.4.0
Requires-Dist: numpy>=1.24.0
Requires-Dist: opencv-python-headless>=4.8.0
Requires-Dist: pillow>=10.0.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: regex>=2023.0.0
Requires-Dist: tqdm>=4.64.0
Provides-Extra: dev
Requires-Dist: pytest-cov>=4.0; extra == 'dev'
Requires-Dist: pytest>=7.0; extra == 'dev'
Requires-Dist: ruff>=0.1.0; extra == 'dev'
Requires-Dist: ty>=0.0.1a1; extra == 'dev'
Provides-Extra: rust
Requires-Dist: khocr-gen-core; extra == 'rust'
Description-Content-Type: text/markdown

# OCR Foundry

Synthetic OCR training data generator with legacy Khmer/English behavior and
opt-in multilingual whole-line rendering. Use `ocr-foundry`; existing `khocr-gen`
commands run the same CLI. The v0.2 distribution `ocr-foundry` installs both
executables. Python imports remain `khocr_gen`, and Rust and `KHOCR_GEN_*`
environment contracts are unchanged.

See [branding and compatibility](https://github.com/0x1d1e/ocr-foundry/blob/main/docs/BRANDING.md)
for migration details. Repository links to `https://github.com/0x1d1e/ocr-foundry`
are valid now.

## Installation

```bash
pip install ocr-foundry
```

`pip install ocr-foundry` is the v0.2 distribution. It provides both the
`ocr-foundry` and `khocr-gen` commands.

The PyPI project `khocr-gen` is deprecated. Release 0.2.0 contains no code and
depends on this package. `khocr-gen==0.1.13` is the last standalone release.
If `khocr-gen==0.1.13` is already installed, upgrade or remove it before
installing `ocr-foundry`, because both distributions would otherwise provide
the `khocr-gen` command.

The `rust` extra is not installable from PyPI because `khocr-gen-core` is not
published. Native use is a source checkout with `uv sync --extra dev --extra rust`.

### Requirements

- Python >= 3.12
- Rust toolchain (`cargo`/`rustc`, edition 2021) - only needed to build the optional acceleration extension

## Quick Start

### 1. Prepare fonts

```text
fonts/
├── khmer/     <- .ttf / .otf Khmer fonts
└── english/   <- .ttf / .otf English fonts
```

Fonts placed directly in `fonts/` (not in a subdirectory) are added to both pools as fallbacks.

### 2. Prepare corpus

A plain UTF-8 text file, one string per line:

```text
សួស្តី
Hello World
ស្វាគមន៍ Welcome
```

### 3. Generate a dataset

```bash
ocr-foundry generate --corpus corpus/corpus.txt --fonts fonts --output data --copies 3 --storage lmdb
```

### 4. View generated images

```bash
# Summary of an LMDB database
ocr-foundry view --lmdb data/train/lmdb

# Extract images
ocr-foundry view --lmdb data/train/lmdb --output-dir extracted

# Show labels only
ocr-foundry view --lmdb data/train/lmdb --labels-only
```

## Multilingual mode and evaluation

`--multilingual` requires Pillow RAQM and renders horizontal logical Unicode lines using
one covering font by default, with opt-in cluster-aware font fallback (`--cluster-font-fallback`)
segmenting grapheme clusters, preserving combining marks and Khmer coeng sequences, and aligning
multi-run segments to a shared baseline. Labels retain normalized logical text without visual reordering.
Arbitrary recursive font pools, explicit NFC/no-normalization policies, and auto/ltr/rtl direction are supported.
JSONL manifests ingest rich metadata, emit `provenance.jsonl` sidecars, and record font hashes in `metadata.jsonl`.

The source recipe `configs/multilingual-realistic.yml` includes clean samples, disables rare oversampling
and foreground crops, and enables a measured line-ink height guard, with opt-in binary glyph-mask guards
(`--min-glyph-height-for-augmentation`) and script/font intensity scaling. Page layout composition
(`ocr-foundry layout`) adds opt-in whitespace word annotations, native RAQM vertical writing (words and
vertical modes requiring shared libraqm with linked FreeType), and paginated table hierarchy. Rendering
checks are not a universal language-quality claim; see the
[capability matrix](https://github.com/0x1d1e/ocr-foundry/blob/main/docs/MULTILINGUAL.md) and
[page layout contract](https://github.com/0x1d1e/ocr-foundry/blob/main/docs/PAGE_LAYOUT.md).

```bash
ocr-foundry evaluate --reference reference.jsonl --predictions predictions.jsonl --normalization NFC
```

The offline evaluator reports corpus CER, exact-line accuracy and reference-language
groups from ID-keyed JSONL. It downloads no model. See the
[schema and frozen real-document protocol](https://github.com/0x1d1e/ocr-foundry/blob/main/docs/EVALUATION.md)
and [provenance/migration roadmap](https://github.com/0x1d1e/ocr-foundry/blob/main/docs/MULTILINGUAL_ROADMAP.md).

## Configuration

All generation parameters can be specified via CLI flags, a YAML config file, or both (CLI overrides YAML).
See [CONFIG.md](https://github.com/0x1d1e/ocr-foundry/blob/main/docs/CONFIG.md) for details.
Example: [generate.yml](https://github.com/0x1d1e/ocr-foundry/blob/main/configs/generate.yml)

Config loading order (lowest to highest priority):
1. argparse built-in defaults
2. YAML config file values
3. Explicit CLI flags

## CLI Reference

See [CLI_REFERENCE.md](https://github.com/0x1d1e/ocr-foundry/blob/main/docs/CLI_REFERENCE.md) for complete command documentation.

## Documentation

- [Getting Started](https://github.com/0x1d1e/ocr-foundry/wiki/Getting-Started) for a first dataset.
- [GitHub wiki](https://github.com/0x1d1e/ocr-foundry/wiki) for usage guides and troubleshooting.
- [In-repository references](https://github.com/0x1d1e/ocr-foundry/blob/main/docs/README.md) for
  the CLI, configuration, augmentation, and Rust extension.

## Acknowledgments

OCR Foundry builds on these open-source libraries:

- [khmernormalizer](https://github.com/seanghay/khmernormalizer) (MIT) - Khmer text normalization
- [OpenCV](https://opencv.org/) (Apache 2.0) - Image processing and augmentation
- [Pillow](https://python-pillow.org/) (HPND) - Font rendering and image creation
- [Albumentations](https://albumentations.ai/) (MIT) - Augmentation primitives
- [NumPy](https://numpy.org/) (BSD-3-Clause) - Numerical operations
- [LMDB](https://www.symas.com/lmdb) (OpenLDAP 2.8) - Embedded database for training pipelines

## License

MIT
