Metadata-Version: 2.4
Name: interscript-ml-tools
Version: 0.1.0
Summary: Unified training framework for Interscript ML models (rababa + secryst)
Author: Interscript Project
License: BSD-3-Clause
Keywords: nlp,transliteration,diacritization,onnx,distillation
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.26
Requires-Dist: pyyaml>=6.0
Requires-Dist: tqdm>=4.66
Requires-Dist: pydantic>=2.5
Provides-Extra: train
Requires-Dist: torch>=2.2; extra == "train"
Requires-Dist: transformers>=4.42; extra == "train"
Requires-Dist: datasets>=2.18; extra == "train"
Requires-Dist: accelerate>=0.27; extra == "train"
Requires-Dist: peft>=0.10; extra == "train"
Requires-Dist: sentencepiece>=0.2; extra == "train"
Provides-Extra: export
Requires-Dist: onnx>=1.16; extra == "export"
Requires-Dist: onnxruntime>=1.17; extra == "export"
Provides-Extra: publish
Requires-Dist: huggingface_hub>=0.23; extra == "publish"
Requires-Dist: omegaconf>=2.3; extra == "publish"
Requires-Dist: pyarrow>=15.0; extra == "publish"
Provides-Extra: sadeed
Requires-Dist: pyarabic>=0.6; extra == "sadeed"
Requires-Dist: prettytable>=3.9; extra == "sadeed"
Requires-Dist: pandas>=2.0; extra == "sadeed"
Requires-Dist: pyarrow>=14.0; extra == "sadeed"
Requires-Dist: huggingface_hub>=0.23; extra == "sadeed"
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: pytest-cov>=5.0; extra == "dev"
Requires-Dist: ruff>=0.4; extra == "dev"
Requires-Dist: mypy>=1.10; extra == "dev"
Dynamic: license-file

# interscript-ml

The **contract** for Interscript's phonological layer — the normative
definition of what a "hidden reading" model is, and the zoo that
publishes models conforming to it.

This repo owns three things and nothing else:

1. **The `models.yaml` index** — the stable URL every runtime resolves
   model ids against (with per-artifact sha256s and split-part support).
2. **The IMF v1 model-zip format** — the artifact contract:
   `metadata.yaml` + ONNX graphs + member sha256 manifest. The normative
   text is [SPEC.md](SPEC.md); the reference loader now lives in the
   [Python crystal](https://github.com/secryst/secryst-py).
3. **The model zoo + publish pipeline** — teachers from
   [secryst-train](https://github.com/secryst/secryst-train)
   are distilled, gated (parity written into the artifact), and released
   as index entries here.

## The system

```
interscript            deterministic transliteration maps + engines
(ruby · js · py)         │ maps that need vocalization dispatch to a
                        │ crystal through stdlib adapters (optional)
                        ▼
secryst crystals       Ruby gem · pip install secryst · npm i secryst
(secryst org)          implement IMF v1 + models.yaml — nothing else
                        │
                        ▼
interscript-ml  ◄────── models/zips resolve through this index
(THIS repo)     ──────► golden sets: crystals diffed against each other

secryst-train          teachers (arabic · persian · urdu + hebrew docs);
(secryst org)          secryst's own product; students ship through the zoo
```

Dependency directions, stated once:

- **interscript-ml depends on nothing.** It is the contract: an index,
  a format, golden sets, and release tooling.
- **Crystals depend only on the contract.** A crystal has zero
  interscript-core dependency — a TTS front-end can phonemize Khmer
  with `pip install secryst` and nothing else.
- **Engines depend on crystals only optionally.** An engine without a
  crystal simply cannot execute maps that declare a vocalization step.
- **Training is owned by secryst.** Teachers live in
  [secryst/secryst-train](https://github.com/secryst/secryst-train) —
  the crystal family's own product. This repo publishes their distilled
  students through the export gate; it does not train anything.

## Repositories

| repo | role |
|---|---|
| [interscript/interscript-ml](https://github.com/interscript/interscript-ml) | this — contract + zoo |
| [secryst/secryst](https://github.com/secryst/secryst) | Ruby crystal (the original, est. 2020) |
| [secryst/secryst-py](https://github.com/secryst/secryst-py) | Python crystal — reference, owns golden generation |
| [secryst/secryst-ts](https://github.com/secryst/secryst-ts) | TypeScript crystal (npm `secryst`) |
| [secryst/secryst.github.io](https://www.secryst.org) | the crystals' documentation site |
| [secryst/secryst-train](https://github.com/secryst/secryst-train) | training monorepo — **secryst-owned teachers** |
| [interscript/rababa](https://github.com/interscript/rababa) · [rababa-farsi](https://github.com/interscript/rababa-farsi) · [rababa-urdu](https://github.com/interscript/rababa-urdu) | archived origins of the train monorepo (full history merged there) |

`runtime/` in this repo is the **frozen origin** of the Python crystal —
kept for provenance; live code and releases are in secryst-py.

## Environment (as implemented by every crystal)

`SECRYST_INDEX` (index URL or path; default: `models.yaml` on this
repo's main) · `SECRYST_CACHE` (default `~/.cache/secryst`). Cache hits
are re-verified against the index on every load.

License: BSD-3-Clause.
