Metadata-Version: 2.4
Name: darija-router
Version: 0.2.0
Classifier: Programming Language :: Rust
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Topic :: Text Processing :: Linguistic
Classifier: Natural Language :: Arabic
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Summary: Moroccan Darija (Arabic) phrase analyzer — classify each word by source language (French/English/Darija), detect semantic entities (money, time, date, phone...) and strip morphological prefixes. Built in Rust for O(1) lookup.
License: MIT
Requires-Python: >=3.8
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Repository, https://github.com/LaamiriOuail/darija-router

# darija-router

Moroccan Darija language router — O(1) phrase analysis with morphological stripping and semantic tagging. Built in Rust, wrapped for Python.

Classifies Arabic-script Darija words by source language (`fr`, `en`, `darija`), Latin spelling, category, probability, and semantic role.

## Install

```bash
pip install darija-router
```

## Quick Start

```python
import darija_router

# "Brother, I want to meet you tomorrow at 3 AM in the square,
#  do you want us to buy a new phone for 500 dirhams?"
result = darija_router.analyze(
    "خويا باغي نلقاك غدوة الساعة 3 صباح فالساحة، "
    "واش باغي نشريو شي تيليفون جديد ب 500 درهم؟"
)
```

**Response:**

```json
[
  {
    "token": "خويا",
    "index": 0,
    "entries": [
      { "lang": "darija", "latin_word": null, "prob": 100.0, "category": "family_relationships" }
    ],
    "stem": null,
    "semantic": null
  },
  {
    "token": "باغي",
    "index": 1,
    "entries": [],
    "stem": null,
    "semantic": null
  },
  {
    "token": "نلقاك",
    "index": 2,
    "entries": [],
    "stem": null,
    "semantic": null
  },
  {
    "token": "غدوة",
    "index": 3,
    "entries": [
      { "lang": "darija", "latin_word": null, "prob": 100.0, "category": "time_weather" }
    ],
    "stem": null,
    "semantic": "DATE"
  },
  {
    "token": "الساعة",
    "index": 4,
    "entries": [
      { "lang": "darija", "latin_word": null, "prob": 100.0, "category": "time_weather" }
    ],
    "stem": {
      "matched_stem": "ساعة",
      "stripped_prefix": "ال",
      "stripped_suffix": null,
      "normalized": false
    },
    "semantic": "TIME"
  },
  {
    "token": "3",
    "index": 5,
    "entries": [],
    "stem": null,
    "semantic": "TIME"
  },
  {
    "token": "صباح",
    "index": 6,
    "entries": [
      { "lang": "darija", "latin_word": null, "prob": 100.0, "category": "time_weather" }
    ],
    "stem": null,
    "semantic": "TIME"
  },
  {
    "token": "فالساحة",
    "index": 7,
    "entries": [],
    "stem": null,
    "semantic": null
  },
  {
    "token": "،",
    "index": 8,
    "entries": [],
    "stem": null,
    "semantic": "PUNCT"
  },
  {
    "token": "واش",
    "index": 9,
    "entries": [
      { "lang": "darija", "latin_word": null, "prob": 100.0, "category": "slang_fillers" }
    ],
    "stem": null,
    "semantic": null
  },
  {
    "token": "باغي",
    "index": 10,
    "entries": [],
    "stem": null,
    "semantic": null
  },
  {
    "token": "نشريو",
    "index": 11,
    "entries": [],
    "stem": null,
    "semantic": null
  },
  {
    "token": "شي",
    "index": 12,
    "entries": [],
    "stem": null,
    "semantic": null
  },
  {
    "token": "تيليفون",
    "index": 13,
    "entries": [
      { "lang": "fr", "latin_word": "téléphone", "prob": 100.0, "category": "technology_digital" }
    ],
    "stem": null,
    "semantic": null
  },
  {
    "token": "جديد",
    "index": 14,
    "entries": [],
    "stem": null,
    "semantic": null
  },
  {
    "token": "ب",
    "index": 15,
    "entries": [],
    "stem": null,
    "semantic": null
  },
  {
    "token": "500",
    "index": 16,
    "entries": [],
    "stem": null,
    "semantic": "MONEY"
  },
  {
    "token": "درهم",
    "index": 17,
    "entries": [
      { "lang": "darija", "latin_word": null, "prob": 100.0, "category": "administration_official" }
    ],
    "stem": null,
    "semantic": "MONEY"
  },
  {
    "token": "؟",
    "index": 18,
    "entries": [],
    "stem": null,
    "semantic": "PUNCT"
  }
]
```

Breakdown:

| Token | Lang | Category | Semantic | Notes |
|-------|------|----------|----------|-------|
| خويا | darija | family | — | exact match — "my brother" |
| باغي | — | — | — | Darija verb (I want) |
| نلقاك | — | — | — | Darija verb (I meet you) |
| غدوة | darija | time | `DATE` | exact match — "tomorrow" |
| الساعة | darija | time | `TIME` | stripped ال → ساعة (hour) |
| 3 | — | — | `TIME` | context: follows الساعة |
| صباح | darija | time | `TIME` | exact match — "morning" |
| فالساحة | — | — | — | Darija (in the square) |
| ، | — | — | `PUNCT` | Arabic comma |
| واش | darija | slang | — | exact match — question particle |
| باغي | — | — | — | Darija verb (you want) |
| نشريو | — | — | — | Darija verb (we buy) |
| شي | — | — | — | Darija (a/some) |
| تيليفون | fr | tech | — | exact match — téléphone |
| جديد | — | — | — | Darija adjective (new) |
| ب | — | — | — | Darija preposition (for/with) |
| 500 | — | — | `MONEY` | context: followed by درهم |
| درهم | darija | admin | `MONEY` | exact match — "dirham" |
| ؟ | — | — | `PUNCT` | Arabic question mark |

## Semantic Tags

Each token gets a `semantic` field with rule-based detection:

| Tag | Detection | Example |
|-----|-----------|---------|
| `PUNCT` | Punctuation characters | `؟` `،` `.` `!` |
| `CARDINAL` | Digits or Arabic number words | `50` `عشرين` `مية` |
| `DECIMAL` | Decimal numbers | `3.5` `١٢.٥` |
| `MONEY` | Cardinal + currency word | `50 درهم` → both tokens tagged |
| `MEASURE` | Cardinal + unit word | `2 كيلو` → both tokens tagged |
| `TIME` | `الساعة` + cardinal, or time words | `الساعة 3` `صباح` `مساء` |
| `DATE` | Date words or cardinal + month | `غدوة` `15 مارس` |
| `ORDINAL` | Arabic ordinal words | `تاني` `تالت` `أول` |
| `FRACTION` | Arabic fraction words | `نص` `ربع` `تلت` |
| `PERCENT` | `%` suffix or بالمئة | `20%` |
| `PHONE` | Phone number patterns | `+212661234567` `0612345678` |
| `EMAIL` | Contains `@` and `.` | `user@gmail.com` |
| `URL` | Starts with `http` or `www.` | `http://google.com` |
| `ADDRESS` | Address keyword + following word | `زنقة الحسن` |

## Single Word Lookup

```python
darija_router.lookup("بون")
```

```json
{
  "token": "بون",
  "index": 0,
  "entries": [
    { "lang": "fr", "latin_word": "bon", "prob": 20.01, "category": "adjectives_expressions" },
    { "lang": "fr", "latin_word": "bonne", "prob": 13.33, "category": "adjectives_expressions" },
    { "lang": "fr", "latin_word": "bon", "prob": 33.33, "category": "money_commerce" },
    { "lang": "fr", "latin_word": "pont", "prob": 33.33, "category": "transportation_vehicles" }
  ],
  "stem": null,
  "semantic": null
}
```

## API Reference

### `darija_router.analyze(phrase: str) -> list[dict]`

Tokenizes a phrase and classifies each token. Returns a list of result objects:

| Field | Type | Description |
|-------|------|-------------|
| `token` | `str` | The original token |
| `index` | `int` | Position in the phrase (0-based) |
| `entries` | `list[dict]` | Classification results (empty if not found) |
| `stem` | `dict or null` | Morphological info if a prefix/suffix was stripped |
| `semantic` | `str or null` | Semantic tag (PUNCT, CARDINAL, MONEY, TIME, DATE, etc.) |

### `darija_router.lookup(word: str) -> dict`

Looks up a single word. Same result format as one element from `analyze()`.

### `darija_router.__version__`

Package version string.

## How It Works

1. **Embedded lexicon** — 1,595 Arabic-script keys compiled into the binary via `include_str!()`. Zero file I/O, zero startup cost.

2. **O(1) HashMap lookup** — exact match against a `HashMap<String, Vec<Entry>>`.

3. **Morphological fallback** — when an exact match fails, the engine strips prefixes and suffixes:

   **Prefixes:** `ال` `و` `ف` `ب` `ل` `م` `ك`

   **Suffixes:** `هم` `ها` `ش` `ك` `ي`

   **Normalization:** `ة` → `ه`

   Tries: exact → normalize → single strip → double strip. Minimum stem length of 2 prevents over-stripping.

4. **Semantic tagger** — rule-based second pass detects 14 semantic categories via regex patterns and context-aware grouping (e.g. cardinal + currency word → `MONEY`).

5. **17 thematic categories** — transportation, technology, food, health, clothing, home, education, sports, work, money, adjectives, family, colors, time, nature, administration, slang.

## Build from Source

```bash
git clone https://github.com/LaamiriOuail/darija-router.git
cd darija-router
pip install maturin
maturin develop --release
```

## License

MIT

