Metadata-Version: 2.4
Name: e2k
Version: 0.8.0
Summary: A tool for automatic English to Katakana conversion
Author-email: Patchethium <asguftuikh@gmail.com>
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy<3.0,>=1.0
Dynamic: license-file

# English to Katakana Translator

[![PyPI version](https://badge.fury.io/py/e2k.svg)](https://badge.fury.io/py/e2k)

`e2k` is a Python library that translates English to Katakana using lightweight RNN models trained on data from Wiktionary and JMdict/EDICT.
It only requires `numpy` as a runtime dependency.
A Rust port with the same models and outputs lives in [`e2k-rs`](./e2k-rs/README.md).

We also provide pretrained models and dictionaries in the **Releases** section (not included in the PyPI package).

## Installation

```bash
pip install e2k
```

## Usage

### 1. Basic Translation

Two models are available:

| Model | Input Type | Description                                  |
| ----- | ---------- | -------------------------------------------- |
| `P2K` | Phonemes   | Converts phonemes (from CMUdict) to Katakana |
| `C2K` | Characters | Converts English text directly to Katakana   |

```python
from e2k import P2K, C2K
from g2p_en import G2p # any g2p library with CMUdict phonemes can work

p2k = P2K()
g2p = G2p()

word = "geogaddi"
print(p2k(g2p(word)))  # ジオダギ

c2k = C2K()
print(c2k(word))       # ジオガディ
```

> [!NOTE]
> While defaulting on `greedy`, both models support `top_k` and `top_p` decoding strategies. It varies the output between each inference, use when you know what you're doing. Pass `seed` to make it reproducible.

```python
c2k(word, "top_p", p=0.9, t=2)
c2k(word, "top_k", k=5, seed=42)
```

Both models also come in a `turbo` variant, a smaller model distilled from the default one. It's about twice as fast with comparable accuracy, though its outputs differ on some words.

```python
c2k = C2K(model="turbo")
p2k = P2K(model="turbo")
```

### 2. Pitch Accent Prediction

The library includes an RNN-based pitch accent predictor trained on [Unidic](https://clrd.ninjal.ac.jp/unidic/).

```python
from e2k import AccentPredictor as Ap, C2K

ap = Ap()
c2k = C2K()

word = "geogaddi"
katakana = c2k(word)
accent = ap(katakana)

print(f"Katakana: {katakana}, Accent: {accent}")
# Katakana: ジオガディ, Accent: 3
```

### 3. Pronunciation Validity (N-Gram Model)

You can check if a word should be transliterated or spelled as-is. A linear SVM over character n-gram scores decides, trained on CMUdict words against synthetic acronyms, abbreviations and random strings.

```python
from e2k import NGram
ngram = NGram()

word = "ussr"
if ngram(word):
    print(c2k(word))
else:
    print(ngram.as_is(word))
```

> Example output:
>
> - `ussr` → ユーエスエスアール
> - `doggy` → ドギー

### 4. Input/Output Symbol Tables

```python
c2k.in_table  # valid characters
p2k.in_table  # valid phonemes
c2k.out_table # katakana output symbols
p2k.out_table # katakana output symbols
accent.in_table # valid katakana symbols for accent prediction
```

> [!WARNING]
> Symbols not in the `in_table` are ignored and may yield unexpected results.

## Performance

| Task                | Model           | Metric     | Score |
| ------------------- | --------------- | ---------- | ----- |
| Katakana Prediction | P2K             | BLEU ↑     | 0.89  |
| Katakana Prediction | C2K             | BLEU ↑     | 0.92  |
| Accent Prediction   | AccentPredictor | Accuracy ↑ | 88.4% |
| Pronunciation Validity | NGram        | Recall ↑   | 98.9% |
| Pronunciation Validity | NGram        | Acronym rejection ↑ | 90.5% |

> [!NOTE]
> The n-gram model is evaluated on held-out CMUdict words for recall, and on the 189 CMUdict acronyms whose pronunciation spells them out letter by letter for rejection.

## Develop

> [!NOTE]
> This section is for development the e2k root package. For training, see [Training](#training)

### Dependencies

We use [`uv`](https://docs.astral.sh/uv/) for dependency management:

- Root package (`e2k` in `src/`) is the PyPI library and only depends on `numpy`.
- Training code lives in the `e2k_train/` workspace member (`e2k_train`) and carries the heavy ML/tooling dependencies.

For root package:

```bash
uv sync
```

### Testing

The model files are tracked in git under `src/e2k/models`, so a fresh clone runs as is.

```bash
# test
uv run src/inference.py
```

## Training

See [here](./e2k_train/README.md) for instructions related to data preparation, training, evaluation and exporting.

## License

- **Code**: [Unlicense](https://unlicense.org/)
- **Dictionaries**: CC-BY-SA (Wiktionary, JMdict/EDICT)
- **Accent Predictor**: Unidic (GPLv2.0/LGPLv2.1/BSD)
- **N-Gram Model**: CMUDict (BSD 2-Clause)

> Machine learning weights may be considered derivative works, check the upstream licenses when distributing.

## Credits

- [Wiktionary](https://www.wiktionary.org/)
- [JMdict / EDICT](http://www.edrdg.org/jmdict/edict.html)
- [Unidic](https://clrd.ninjal.ac.jp/unidic/)
- [CMUDict](https://github.com/cmusphinx/cmudict)
