Metadata-Version: 2.4
Name: darija-tools
Version: 0.2.0
Summary: Small, honest NLP utilities for Moroccan Darija: normalization and Arabizi transliteration.
Author: Sami El Akkad
License-Expression: MIT
Project-URL: Homepage, https://github.com/Samielakkad/darija-tools
Project-URL: Source, https://github.com/Samielakkad/darija-tools
Project-URL: Issues, https://github.com/Samielakkad/darija-tools/issues
Project-URL: Changelog, https://github.com/Samielakkad/darija-tools/blob/main/CHANGELOG.md
Project-URL: Documentation, https://github.com/Samielakkad/darija-tools#readme
Keywords: darija,moroccan-arabic,arabic,nlp,transliteration,arabizi
Classifier: Development Status :: 3 - Alpha
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Topic :: Text Processing :: Linguistic
Classifier: Natural Language :: Arabic
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: dev
Requires-Dist: build>=1.2; extra == "dev"
Requires-Dist: pytest<9,>=8; extra == "dev"
Requires-Dist: ruff>=0.11; extra == "dev"
Requires-Dist: twine<7,>=6; extra == "dev"
Dynamic: license-file

# darija-tools

[![CI](https://github.com/Samielakkad/darija-tools/actions/workflows/ci.yml/badge.svg)](https://github.com/Samielakkad/darija-tools/actions/workflows/ci.yml)
[![PyPI](https://img.shields.io/pypi/v/darija-tools)](https://pypi.org/project/darija-tools/)
[![License: MIT](https://img.shields.io/badge/license-MIT-green)](LICENSE)

Small, dependency-free text utilities for Moroccan Darija (الدارجة). The rules and word lists are plain files that can be reviewed and tested.

## Install

```bash
pip install darija-tools
```

Python 3.9 or newer is supported.

## Use it

```python
from darija_tools import normalize, to_arabic, to_arabizi

normalize("الأَحْمَر")
# 'الاحمر'

to_arabic("3lach bghiti daba")
# 'علاش بغيتي دبا'

to_arabizi("علاش بغيتي دبا")
# '3lach bghiti daba'
```

`to_arabic()` checks a curated Darija word list, then uses letter and digraph rules for unknown words. Set `keep_loanwords=True` to leave recognized French and English loanwords in Latin script:

```python
to_arabic("bghit taxi", keep_loanwords=True)
# 'بغيت taxi'
```

`to_arabizi()` uses the same lexicon in reverse and chooses one stable spelling when a word has several valid Arabizi forms. Unknown Arabic words use a readable character mapping.

## Command line

```bash
darija normalize "الأَحْمَر"
darija translit "3lach bghiti daba"
darija translit --keep-loanwords "bghit taxi"
darija arabizi "علاش بغيتي دبا"
```

Omit the text argument to read from standard input.

## Evaluation

The v0.2 Arabizi-to-Arabic evaluation contains 30 phrases kept outside the unit tests:

- Phrase exact match: **26/30 (86.7%)**
- Whitespace-token exact match: **61/66 (92.4%)**

The [dataset](https://github.com/Samielakkad/darija-tools/blob/v0.2.0/evaluation/held_out_v0.2.jsonl), [full error report](https://github.com/Samielakkad/darija-tools/blob/v0.2.0/evaluation/report-v0.2.md) and [evaluation script](https://github.com/Samielakkad/darija-tools/blob/v0.2.0/evaluation/run.py) are checked in. CI regenerates the result and fails if the committed report differs.

This is a small, manually curated regression set, not a blinded study or a claim about all Darija text. The four failed phrases remain visible in the report.

## Limits

- Arabizi spelling is not standardized, so transliteration is lossy in both directions.
- Character rules cannot reliably recover unwritten vowels, doubled consonants or word endings.
- Loanword preservation covers a small reviewed list and is opt-in.
- The target is Moroccan Darija; overlap with other Maghrebi dialects is incidental.

## Development

```bash
python -m pip install -e ".[dev]"
python -m ruff check src tests evaluation
python -m pytest -q
python evaluation/run.py --check
python -m build
python -m twine check dist/*
```

See [CONTRIBUTING.md](https://github.com/Samielakkad/darija-tools/blob/main/CONTRIBUTING.md) before changing language rules or data. v0.2 changes are summarized in the [release notes](https://github.com/Samielakkad/darija-tools/blob/v0.2.0/docs/release-v0.2.md).

## License

MIT. See [LICENSE](https://github.com/Samielakkad/darija-tools/blob/main/LICENSE).
