Metadata-Version: 2.4
Name: arabic-morph-segmenter
Version: 0.1.0
Summary: Lightweight Arabic morphological prefix-stem-suffix segmenter using BiLSTM-CRF.
Author: Faisal Alshargi
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: torch
Requires-Dist: pytorch-crf
Dynamic: license-file

# Arabic Morph Segmenter

> A lightweight neural Arabic morphological segmenter based on **BiLSTM + CRF** for **Modern Standard Arabic (MSA)** and multiple **Arabic dialects**.

![Python](https://img.shields.io/badge/Python-3.9+-blue.svg)
![PyTorch](https://img.shields.io/badge/PyTorch-2.x-red.svg)
![License](https://img.shields.io/badge/License-MIT-green.svg)

---

## Overview

Arabic words frequently contain attached conjunctions, prepositions, articles, pronouns, and other clitics.

For example,

```
وبكتابهم
```

should be segmented into

```
وب + كتاب + هم
```

instead of being treated as a single token.

This project provides a lightweight neural segmenter that predicts the character-level boundaries of:

- Prefix
- Stem
- Suffix

using a **BiLSTM-CRF** architecture.

The output can be used as a preprocessing step for:

- POS Tagging
- Named Entity Recognition (NER)
- Lemmatization
- Dependency Parsing
- Arabic Search
- Information Retrieval
- Large Language Models (LLMs)
- Arabic NLP pipelines

---

# Features

- Lightweight (~9 MB)
- Fast inference
- Character-level neural segmentation
- BiLSTM + CRF architecture
- Supports Modern Standard Arabic
- Supports multiple Arabic dialects
- Easy Python API
- Ready for PyPI

---

# Supported Arabic

Current training includes:

- Modern Standard Arabic (MSA)
- Iraqi Arabic
- Syrian Arabic
- Moroccan Arabic
- Najdi Arabic
- Sanaani Arabic
- Taizi Arabic

The architecture is language-independent and can easily be extended with additional dialects.

---

# Model Architecture

```
Characters
      │
      ▼
Character Embedding
      │
      ▼
2-layer BiLSTM
      │
      ▼
Linear Layer
      │
      ▼
CRF
      │
      ▼
P / S / X
```

Where

- **P** = Prefix
- **S** = Stem
- **X** = Suffix

---

# Installation

```bash
pip install arabic-morph-segmenter
```

---

# Quick Start

```python
from arabic_morph_segmenter import ArabicMorphSegmenter

segmenter = ArabicMorphSegmenter()

result = segmenter.segment_word("وبكتابهم")

print(result)
```

Output

```python
{
    "word": "وبكتابهم",
    "prefix": "وب",
    "stem": "كتاب",
    "suffix": "هم",
    "segmented": "وب + كتاب + هم"
}
```

---

# Sentence Segmentation

```python
sentence = "والناس تفرقت من المدرسة"

for token in segmenter.segment_sentence(sentence):
    print(token["segmented"])
```

Output

```
وال + ناس
تفرق + ت
من
ال + مدرس + ة
```

---

# More Examples

| Word | Segmentation |
|------|--------------|
| وندد | و + ندد |
| الفلسطينيون | ال + فلسطين + يون |
| وبكتابهم | وب + كتاب + هم |
| فبالمدرسة | فبال + مدرس + ة |
| والناس | وال + ناس |
| تفرقت | تفرق + ت |
| وسيكتبون | وسي + كتب + ون |
| وللمهندسين | ولل + مهندس + ين |

---

# Evaluation

Evaluation on the held-out validation set:

| Metric | Score |
|---------|-------|
| Character Accuracy | **99.04%** |
| Word Exact Accuracy | **96.90%** |

---

# Training Data

The model was trained on a combination of:

- Modern Standard Arabic morphological corpus
- Iraqi dialect
- Syrian dialect
- Moroccan dialect
- Najdi dialect
- Sanaani dialect
- Taizi dialect

using character-level prefix/stem/suffix annotations.

---

# Project Structure

```
arabic_morph_segmenter/
│
├── assets/
│   └── arabic_morph_segmenter_bilstm_crf.pt
│
├── segmenter.py
├── __init__.py
│
examples/
│   └── demo.py
│
README.md
LICENSE
```

---

# Citation

If you use this project in your research, please cite:

```bibtex
@software{alshargi2026arabicsegmenter,
  author = {Faisal Alshargi},
  title = {Arabic Morph Segmenter},
  year = {2026},
  url = {https://github.com/alshargi/arabic-morph-segmenter}
}
```

---

# Roadmap

- ONNX export
- Hugging Face model
- REST API
- CLI interface
- Transformer version
- Additional Arabic dialects
- Confidence scores
- Batch inference

---

# License

MIT License.

---

# Author

**Dr. Faisal Alshargi**

AI • NLP • Arabic Language Technologies • Large Language Models

GitHub: https://github.com/alshargi

---

⭐ If you find this project useful, please consider giving it a star.
