Metadata-Version: 2.4
Name: darija-router
Version: 0.6.1
Classifier: Programming Language :: Rust
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Topic :: Text Processing :: Linguistic
Classifier: Natural Language :: Arabic
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Summary: Moroccan Darija (Arabic) phrase analyzer — classify each word by source language (French/English/Darija), detect semantic entities (money, time, date, phone...) and strip morphological prefixes. Built in Rust for O(1) lookup.
License: MIT
Requires-Python: >=3.8
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Repository, https://github.com/LaamiriOuail/darija-router

# darija-router

Moroccan Darija language router — O(1) phrase analysis with morphological stripping, semantic tagging, NER, and code-switching detection. Built in Rust, wrapped for Python.

Classifies Arabic-script Darija words by source language (`fr`, `en`, `darija`), Latin spelling, category, probability, semantic role, and confidence score.

## Install

```bash
pip install darija-router
```

## Quick Start

```python
import darija_router

result = darija_router.analyze(
    "خويا باغي نلقاك غدوة الساعة 3 صباح فالساحة، "
    "واش باغي نشريو شي تيليفون جديد ب 500 درهم؟"
)
```

## API Reference

### `darija_router.analyze(phrase, mode=None, fields=None)`

Tokenizes a phrase and classifies each token.

**Parameters:**
- `phrase` (str): Input text in Arabic script
- `mode` (str, optional): Output preset — `"full"` (default), `"compact"`, `"ner"`, `"codeswitch"`
- `fields` (list[str], optional): Custom field selection — e.g. `["token", "lang", "confidence"]`

**Returns:** `list[dict]` with per-token results.

### `darija_router.run_ner(phrase)`

Named Entity Recognition. Returns span-based entities and per-token NER tags.

**Returns:**
```json
{
  "entities": [
    {"text": "500 درهم", "entity_type": "MONEY", "start": 0, "end": 8, "token_indices": [0, 1]}
  ],
  "tokens": [
    {"token": "500", "ner_tag": "MONEY"},
    {"token": "درهم", "ner_tag": "MONEY"}
  ]
}
```

**Entity types:** MONEY, MEASURE, TIME, DATE, PHONE, EMAIL, URL, CARDINAL, DECIMAL, ORDINAL, FRACTION, PERCENT, PERSON, LOC

### `darija_router.run_codeswitch(phrase)`

Code-switching detection with language spans and statistics.

**Returns:**
```json
{
  "spans": [
    {"text": "خويا واش", "lang": "darija", "start": 0, "end": 9}
  ],
  "tokens": [
    {"token": "خويا", "lang": "darija", "latin": null, "confidence": 1.0}
  ],
  "stats": {
    "darija": 50.0, "fr": 25.0, "en": 0.0, "num": 12.5, "punct": 0.0, "other": 0.0, "unknown": 12.5
  }
}
```

### `darija_router.lookup(word)`

Single word lookup. Same format as one element from `analyze()`.

### `darija_router.__version__`

Package version string.

## Semantic Tags

| Tag | Detection | Example |
|-----|-----------|---------|
| `PUNCT` | Punctuation characters | `؟` `،` `.` `!` |
| `CARDINAL` | Digits or Arabic number words | `50` `عشرين` `مية` |
| `DECIMAL` | Decimal numbers | `3.5` `١٢.٥` |
| `MONEY` | Cardinal + currency word | `50 درهم` → both tokens tagged |
| `MEASURE` | Cardinal + unit word | `2 كيلو` → both tokens tagged |
| `TIME` | `الساعة` + cardinal, or time words | `الساعة 3` `صباح` `مساء` |
| `DATE` | Date words or cardinal + month | `غدوة` `15 مارس` |
| `ORDINAL` | Arabic ordinal words | `تاني` `تالت` `أول` |
| `FRACTION` | Arabic fraction words | `نص` `ربع` `تلت` |
| `PERCENT` | `%` suffix or بالمئة | `20%` |
| `PHONE` | Phone number patterns | `+212661234567` |
| `EMAIL` | Contains `@` and `.` | `user@gmail.com` |
| `URL` | Starts with `http` or `www.` | `http://google.com` |
| `ADDRESS` | Address keyword + following word | `زنقة الحسن` |

## Output Modes

| Mode | Description |
|------|-------------|
| `"full"` (default) | All fields: token, index, lang, entries, stem, semantic, confidence |
| `"compact"` | token + lang only (no entries/stem) |
| `"ner"` | NER preset: entities + tokens with ner_tag |
| `"codeswitch"` | Code-switching preset: spans + tokens + stats |

## Confidence Scoring

| Match Type | Score |
|------------|-------|
| Exact match | 1.0 |
| Normalized (ة→ه) | 0.95 |
| Single prefix/suffix strip | 0.85 |
| Double strip (prefix + suffix) | 0.70 |
| Semantic tag only (no lexicon match) | 0.60 |
| Unknown | 0.0 |

## How It Works

1. **Embedded lexicon** — 1,926 Arabic-script keys (2,226 entries) compiled into the binary via `include_str!()`. Zero file I/O, zero startup cost.

2. **O(1) HashMap lookup** — exact match against a `HashMap<String, Vec<Entry>>`.

3. **Morphological fallback** — when an exact match fails, the engine strips prefixes and suffixes:

   **Prefixes:** `ال` `و` `ف` `ب` `ل` `م` `ك`

   **Suffixes:** `هم` `ها` `ش` `ك` `ي`

   **Normalization:** `ة` → `ه`

   Tries: exact → normalize → single strip → double strip. Minimum stem length of 2 prevents over-stripping.

4. **Semantic tagger** — rule-based second pass detects 14 semantic categories via regex patterns and context-aware grouping (e.g. cardinal + currency word → `MONEY`).

5. **NER engine** — extracts span-based entities with context rules for PERSON (سيدي/لالة + next word) and LOC (شارع/زنقة/حي + next words).

6. **Code-switching detector** — assigns per-token language labels, groups consecutive same-language tokens into spans, and computes aggregate statistics.

7. **23 thematic categories** — technology, food, transportation, health, administration, clothing, home, education, sports, work, money, adjectives, family, colors, time, nature, slang, functional grammar, common verbs, common descriptions, time expressions, numbers.

## Build from Source

```bash
git clone https://github.com/LaamiriOuail/darija-router.git
cd darija-router
pip install maturin
maturin develop --release
```

## License

MIT

