Metadata-Version: 2.5
Name: tetrak-translit
Version: 4.3.0
Summary: Detect, transliterate and fold Armenian, Georgian and Russian text — ALA-LC, ISO 9985 and ISO 9984, BGN/PCGN, the Georgian national system, Hübschmann-Meillet and Western pronunciation, plus one canonical index form for cross-script search
Project-URL: Homepage, https://github.com/scattercode/tetrak-translit
Project-URL: Documentation, https://scattercode.github.io/tetrak-translit/
Project-URL: Tetrak, https://tetrak.dev/
Project-URL: Source, https://github.com/scattercode/tetrak-translit
Project-URL: Changelog, https://github.com/scattercode/tetrak-translit/blob/main/CHANGELOG.md
Project-URL: Issues, https://github.com/scattercode/tetrak-translit/issues
Author: Stephen Masters, Yvette Mankerian
License-Expression: MIT
License-File: LICENSE
Keywords: ala-lc,archives,armenian,bgn-pcgn,cyrillic,georgian,iso-9984,iso-9985,libraries,romanisation,romanization,russian,script-detection,search,transliteration
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Information Technology
Classifier: Intended Audience :: Science/Research
Classifier: Natural Language :: Armenian
Classifier: Natural Language :: Georgian
Classifier: Natural Language :: Russian
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Text Processing :: Indexing
Classifier: Topic :: Text Processing :: Linguistic
Classifier: Typing :: Typed
Requires-Python: >=3.10
Provides-Extra: apidocs
Requires-Dist: furo>=2024.8; extra == 'apidocs'
Requires-Dist: myst-parser>=4.0; extra == 'apidocs'
Requires-Dist: sphinx-argparse>=0.6.1; extra == 'apidocs'
Requires-Dist: sphinx>=8.0; extra == 'apidocs'
Provides-Extra: dev
Requires-Dist: mypy>=1.10; extra == 'dev'
Requires-Dist: pytest-cov>=5.0; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff>=0.5; extra == 'dev'
Description-Content-Type: text/markdown

# tetrak-translit

Detect, transliterate and fold text in scripts that have several
romanisations: Armenian, Georgian and Russian so far. A dependency-free Python
library and command-line tool for the problem every catalogue of such
material has: the same name exists in its own script, sometimes in more
than one orthography, and in three or five Latin spellings, and no stock
search engine treats them as one word.

> **Status: stable.** The API has not changed since 1.0; the major
> releases since followed new editions of the standards, which moved
> renderings. Versions say what moved: a change to what any input renders
> or folds to is at least a minor release, since indexes built on the fold
> must be rebuilt, and a change to an existing test case is a major one. The tables and the folds
> are tested against the sets in [`tests/data/`](tests/data/), hundreds of
> Armenian fold cases among them; what is not in them is not claimed, and
> the [known gaps](#what-it-does-not-do) are written down. Measured in
> catalogue search across scripts over about a thousand records, romanised
> queries in every scheme now fold to the script's key 99 to 100 times in
> a hundred, up from 82 at the first release.

```bash
pip install tetrak-translit
```

```python
>>> import tetrak_translit as tt
>>> tt.detect("Երեւան").orthography
'classical'
>>> tt.transliterate("Երևան", "ala_lc")
'Erevan'
>>> tt.transliterate("Երևան", "bgn_pcgn")
'Yerevan'
>>> tt.transliterate("Յակոբ", "western")
'Hagop'
>>> tt.transliterate("თბილისი", "national")
'tbilisi'
>>> tt.fold("Yerevan", script="hy") == tt.fold("Երեւան") == tt.fold("Erewan", script="hy")
True
>>> tt.fold("Boghossian", script="hy") == tt.fold("Պօղոսեան") == tt.fold("Poghosyan", script="hy")
True
>>> tt.fold("Chavchavadze", script="ka") == tt.fold("ჭავჭავაძე") == tt.fold("Ch'avch'avadze", script="ka")
True
>>> tt.detect("Русскій вѣстникъ").orthography
'pre-reform'
>>> tt.fold("Tchaikovsky", script="ru") == tt.fold("Чайковский") == tt.fold("Chaĭkovskiĭ", script="ru")
True
```

```bash
$ tetrak-translit detect "Ազգ եւ հայրենիք" "თბილისი"
armenian (classical)	Ազգ եւ հայրենիք
georgian	თბილისი
$ tetrak-translit transliterate --scheme iso_9985 "Չարենց"
Č'arenc'
$ tetrak-translit transliterate --scheme iso_9984 --to script "Tbilisi"
თბილისი
$ cut -f2 titles.tsv | tetrak-translit fold --script hy
```

## Three jobs

**Detection.** Which script a string is in and, for Armenian, which
orthography: *classical* (everything printed before the 1922 reform, and
the diaspora still), *reformed* (Soviet and post-Soviet Armenia) or
*indeterminate* (the string reads the same under both). For Russian,
*pre-reform* (the four letters and the final hard sign the 1918 reform
removed, which émigré presses kept) or *reformed*. Unicode character
names plus a few orthographic tells, chiefly a free `ւ` outside the `ու`
digraph. For catalogue search across scripts, where a title's language
code often says nothing, a Latin value is also read for the marks a
romanisation leaves: the aspirate in *Patmutʻiwn*, the *ĕ* of *Keankʻi
mĕ*. Detection reports the marks it finds as evidence, never as a verdict.

**Transliteration**, between a script and its named Latin schemes, each a
versioned, tested table written from its published standard:

| Id | Scheme | Where you meet it | Reversible |
|---|---|---|---|
| `hy/ala_lc` | ALA-LC Armenian (2022) | Library catalogues: Koha, WorldCat, the Library of Congress | no |
| `hy/iso_9985` | ISO 9985:2026 | Data exchange; one letter per letter | **yes** |
| `hy/iso_9985_1996` | ISO 9985:1996, superseded | Reading data romanised under the first edition | **yes** |
| `hy/bgn_pcgn` | BGN/PCGN 1981 | Maps and place names: *Yerevan*, *Gyumri* | no |
| `hy/hubschmann_meillet` | Hübschmann-Meillet | Linguistics and classical scholarship | **yes** |
| `hy/western` | Western Armenian pronunciation | Diaspora names: *Hagop*, *Kevork*, *Boghosian* | no |
| `ka/ala_lc` | ALA-LC Georgian (2011) | Library catalogues | no |
| `ka/iso_9984` | ISO 9984:2026 | Data exchange; the national system made reversible | **yes** |
| `ka/iso_9984_1996` | ISO 9984:1996, superseded | Reading data romanised under the first edition | **yes** |
| `ka/national` | Georgian national system (2002) | Road signs, passports, maps: *Tbilisi*, *Kutaisi* | no |
| `ru/ala_lc` | ALA-LC Russian (2012), with the pre-1918 letters | Library catalogues: *Chaĭkovskiĭ*, *vi͡estnikʺ* | no |
| `ru/bgn_pcgn` | BGN/PCGN 1947 | Maps and place names: *Yekaterinburg* | no |

A scheme key alone is enough when the text or the key settles the script;
otherwise say `script="ka"` or use the qualified id. Reversible where the
standard is one-to-one; the others are documented as lossy and the library
refuses to guess rather than reverse them.

**The canonical fold.** One function that maps any of the above, script
or Latin, onto a single index form, so that index-time and query-time
normalisation agree. This is what a Solr or Elasticsearch analysis chain
calls. Each script owns its rules, because what should be merged differs
(Armenian folds *b* with *p*, Georgian must not), so Latin input names
the script it is to be folded for. The Armenian fold generalises the
script fold in Tetrak's Armenian recogniser and takes the orthography
findings of Tetrak brief 014 as its first test cases.

The schemes are for **display**; the fold is for **finding**. A fold key is
a deliberately coarse ASCII skeleton (`poghosian`, `kevork`, `haiastan`)
and must never be shown to a reader. Cross-script name-matching research
(LREC 2020) finds there is often no gold form to score against, and this
package does not pretend there is one.

## What it does not do

- **Translate.** Transliteration is the same words in different letters.
- **Guess at lossy reversals.** `transliterate(..., to="script")` works for
  ISO 9985, Hübschmann-Meillet and ISO 9984 and raises for the rest.
- **Fold every popular spelling.** Spellings that insert an unwritten
  schwa (*Megerdich* for `Մկրտիչ`), come through Russian (*Khachaturian*
  for `Խաչատրյան`, *Tiflis* for `თბილისი`) or are translations (*George*
  for `გიორგი`) do not fold to the script, nor do the German and French
  spellings of Russian names that use a letter for another sound
  (*Michail*, *Zwetajewa*, *Pouchkine*), and a classical Armenian `աւ`
  that reformed spelling turned into `օ` does not either. The test sets
  are the claim.
- **Handle every script.** Armenian, Georgian and Russian so far; Syriac
  is next. The engine is
  script-agnostic and a new script is a package of tables; see
  [adding a script](https://scattercode.github.io/tetrak-translit/adding-a-script.html).

## Documentation

The user guide and API reference are at
[scattercode.github.io/tetrak-translit](https://scattercode.github.io/tetrak-translit/),
built from the released code at each release.

## Where it comes from

Built as part of [Tetrak](https://tetrak.dev/), a local-first OCR pipeline
for archival material, to support two use cases at once: a catalogue search that has to match records across scripts, and Tetrak's
own transcripts, where an ALA-LC rendering beside the OCR output is both a
catalogue-ready field and an accessibility one (a screen reader with no
Armenian voice can at least read the romanisation). Georgian came next
because Tetrak's recognisers are heading there; Syriac is planned.

## Contributing and licence

See [CONTRIBUTING.md](CONTRIBUTING.md) for the checks, the commit style and
the release automation, and [SECURITY.md](SECURITY.md) for reporting a
vulnerability. MIT licensed — see [LICENSE](LICENSE).
