Metadata-Version: 2.5
Name: tetrak-hy-trainer
Version: 0.2.0
Summary: Training pipeline for an Armenian text recogniser, packaged as an EasyOCR custom model
Project-URL: Homepage, https://github.com/scattercode/tetrak-hy-trainer
Project-URL: Tetrak, https://tetrak.dev/
Project-URL: Source, https://github.com/scattercode/tetrak-hy-trainer
Project-URL: Changelog, https://github.com/scattercode/tetrak-hy-trainer/blob/main/CHANGELOG.md
Project-URL: Issues, https://github.com/scattercode/tetrak-hy-trainer/issues
Author: Stephen Masters, Yvette Mankerian
License-Expression: Apache-2.0
License-File: LICENSE
License-File: NOTICE
Keywords: archives,armenian,easyocr,ocr,text-recognition
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Natural Language :: Armenian
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Scientific/Engineering :: Image Recognition
Classifier: Topic :: Text Processing
Requires-Python: >=3.11
Requires-Dist: pyyaml>=6.0
Requires-Dist: requests>=2.31
Provides-Extra: dev
Requires-Dist: pillow>=10.0; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff>=0.5; extra == 'dev'
Provides-Extra: train
Requires-Dist: natsort>=8.0; extra == 'train'
Requires-Dist: nltk>=3.8; extra == 'train'
Requires-Dist: numpy>=1.24; extra == 'train'
Requires-Dist: opencv-python-headless>=4.6; extra == 'train'
Requires-Dist: pandas>=1.5; extra == 'train'
Requires-Dist: pillow>=10.0; extra == 'train'
Requires-Dist: six>=1.16; extra == 'train'
Requires-Dist: torch>=2.0; extra == 'train'
Requires-Dist: torchvision>=0.15; extra == 'train'
Description-Content-Type: text/markdown

# tetrak-hy-trainer

Training pipeline for an Armenian text recogniser — synthetic data
generation, CTC model training, and packaging as an
[EasyOCR](https://github.com/JaidedAI/EasyOCR) custom model.

## Why

No mainstream local OCR engine reads Armenian well. EasyOCR and PaddleOCR do
not list the language at all; Tesseract ships `hye` traineddata of
unmeasured quality on archival material. Yet the architecture EasyOCR
already uses — CRAFT text detection feeding a compact CTC recogniser — is
proven on Armenian: a National Library of Armenia-adjacent system built on
exactly these components reported character error rates better than Google
Cloud Vision on dense newsprint.

Detection needs no training (CRAFT is script-agnostic). All the
Armenian-specific work concentrates in one small trainable model, and
EasyOCR has a documented custom-model mechanism to load it. This repository
builds that model.

## What it produces

Three files, loadable by stock EasyOCR:

| File | Contents |
|---|---|
| `tetrak_hy.yaml` | Character list, language list, image height, network parameters |
| `tetrak_hy.py` | The recognition network module (`Model(num_class, **network_params)`) |
| `tetrak_hy.pth` | Trained weights — published as GitHub Release assets, never committed |

```python
import easyocr

reader = easyocr.Reader(
    ["hy"],
    recog_network="tetrak_hy",
    user_network_directory="path/holding/yaml/and/py",
    model_storage_directory="path/holding/pth",
)
reader.readtext("scan.png")
```

The name is a Python module name (EasyOCR imports it), hence the
underscore. The model uses a CTC head — EasyOCR's custom-model inference
path is CTC-only.

**Distribution happens through
[tetrak-easyocr-armenian](https://github.com/scattercode/tetrak-easyocr-armenian)**,
the installable library whose import package *is* the `tetrak_hy` network:
this trainer's output — the yaml, the weights and their provenance — lands
there as a pull request, and a weights release on that repo is what users
install. The bundle-writing here remains the tool for local spikes and for
producing that PR.

## Status

Early scaffold. The pipeline stages, in order:

1. **Charset** — `src/tetrak_hy_trainer/charset.py`, the single source of
   truth read by both the trainer and the packaging step. ✔ (two decisions
   deliberately open; see the module)
2. **Packaging** — emit a valid `tetrak_hy.yaml` from the charset. ✔
3. **Spike** — train a deliberately tiny model and prove the EasyOCR
   loading contract end to end. Not started.
4. **Synthetic data** — Armenian corpus text rendered in Armenian fonts
   with archival degradations. Not started.
5. **Training** — CTC pre-training on synthetic crops, fine-tuning on
   human-verified real crops. Not started.

## Data and font licences

Recorded as sources are adopted:

| Source | Use | Licence |
|---|---|---|
| [Armenian Soviet Encyclopedia](https://hy.wikisource.org/wiki/Հայկական_սովետական_հանրագիտարան) on Armenian Wikisource (13 volumes, 1974–1987) | Corpus text for synthesis; paired page scans + transcripts for fine-tuning crops and evaluation | CC BY-SA 3.0, as stated by the hosting Wikisource page |

Two disciplines attach to the encyclopedia source:

- **Only proofread pages.** Wikisource seeds unproofread pages with
  machine OCR; training on those would teach the model another engine's
  mistakes. Harvesting filters on ProofreadPage quality status
  (proofread/validated only), via the API.
- **Attribution and share-alike.** BY is satisfied by this table and the
  provenance records shipped with weights. Whether SA obligations
  propagate to trained weights is a genuinely unsettled question; we
  record the source and licence with every release so the position is
  auditable either way.

Further candidates: Armenian Wikisource's public-domain period texts and
Armenian Wikipedia (CC BY-SA 4.0) for corpus text; the Noto Armenian
family, GHEA faces and Arian AMU (all OFL) for fonts.

## Licence

Apache License 2.0 — see
[LICENSE](https://github.com/scattercode/tetrak-hy-trainer/blob/main/LICENSE)
and
[NOTICE](https://github.com/scattercode/tetrak-hy-trainer/blob/main/NOTICE).
Training code will derive in part from EasyOCR's trainer (Apache 2.0), itself
derived from NAVER's
[deep-text-recognition-benchmark](https://github.com/clovaai/deep-text-recognition-benchmark)
(Apache 2.0).

**A deliberate exclusion:** this project was informed by studying
[portmind/armenian-ocr](https://github.com/portmind/armenian-ocr)
(CC BY-NC 4.0), whose approach it independently reproduces from
permissively-licensed parts. No code, annotations or weights from that
project are included here, and contributions derived from it cannot be
accepted — its non-commercial licence is incompatible with this one.

## Relationship to Tetrak

This is a satellite of [Tetrak](https://tetrak.dev/), a local-first
transcription pipeline for archival material. Tetrak ships the inference
files and consumes the released weights as its `easyocr-hy` backend;
benchmark results against its evaluation corpus are published there.

## Development

```bash
python3 -m venv .venv && source .venv/bin/activate
pip install -e '.[dev]'
pytest
ruff check src tests && ruff format --check src tests
```

Commits follow [Conventional Commits](https://www.conventionalcommits.org/),
enforced by the hook in `.githooks/` (`git config core.hooksPath .githooks`
after cloning, or `lefthook install`). Releases and `CHANGELOG.md` are
generated from those commits automatically on every push to `main`.

See
[CONTRIBUTING.md](https://github.com/scattercode/tetrak-hy-trainer/blob/main/CONTRIBUTING.md)
for the full workflow — checks, the dependency lockfile, and what the
automation expects — and
[SECURITY.md](https://github.com/scattercode/tetrak-hy-trainer/blob/main/SECURITY.md)
for how to report a vulnerability.
