Metadata-Version: 2.4
Name: arabic-survey-analyzer
Version: 0.1.0
Summary: Smart Arabic survey analysis: Likert, reliability, validity, inferential tests, open-text analysis and automatic Arabic Word/Excel reports
Author-email: Dr Merwan Roudane <merwanroudane920@gmail.com>
License-Expression: MIT
Project-URL: Homepage, https://github.com/merwanroudane/surveyarabic
Project-URL: Repository, https://github.com/merwanroudane/surveyarabic
Project-URL: Issues, https://github.com/merwanroudane/surveyarabic/issues
Keywords: survey,arabic,likert,cronbach-alpha,mcdonald-omega,questionnaire,statistics,reliability,validity,factor-analysis,sentiment-analysis,rtl,word-report
Classifier: Development Status :: 4 - Beta
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Natural Language :: Arabic
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: Education
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Classifier: Operating System :: OS Independent
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pandas>=1.5
Requires-Dist: numpy>=1.23
Requires-Dist: scipy>=1.9
Requires-Dist: statsmodels>=0.13
Requires-Dist: factor_analyzer>=0.4
Requires-Dist: matplotlib>=3.5
Requires-Dist: openpyxl>=3.0
Requires-Dist: python-docx>=0.8.11
Requires-Dist: arabic-reshaper>=2.1
Requires-Dist: python-bidi>=0.4
Provides-Extra: app
Requires-Dist: streamlit>=1.30; extra == "app"
Requires-Dist: plotly>=5.0; extra == "app"
Provides-Extra: ai
Requires-Dist: anthropic>=0.40; extra == "ai"
Dynamic: license-file

# ArabicSurveyAnalyzer — محلل الاستبيانات العربية

[![Repo](https://img.shields.io/badge/GitHub-merwanroudane%2Fsurveyarabic-1f3864)](https://github.com/merwanroudane/surveyarabic)
[![Python](https://img.shields.io/badge/python-%E2%89%A53.9-blue)](https://www.python.org/)
[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)

مكتبة Python ذكية لتحليل الاستبيانات باللغة العربية: من ملف Excel/CSV إلى
تقرير عربي أكاديمي كامل (Word + Excel + JSON + رسوم بيانية) في سطور قليلة.

A smart Python library for Arabic-language survey analysis: from a raw
Excel/CSV file to a complete academic Arabic report (Word + Excel + JSON +
publication-quality charts) in a few lines of code.

**Author:** Dr Merwan Roudane — <https://github.com/merwanroudane/surveyarabic>

---

## Table of contents — المحتويات

1. [Features — المزايا](#1-features--المزايا)
2. [Installation — التثبيت](#2-installation--التثبيت)
3. [Input data format — صيغة البيانات](#3-input-data-format--صيغة-البيانات)
4. [Quick start — البداية السريعة](#4-quick-start--البداية-السريعة)
5. [`SurveyAnalyzer` — full API reference](#5-surveyanalyzer--full-api-reference)
6. [Results objects — بنية النتائج](#6-results-objects--بنية-النتائج)
7. [Outputs — المخرجات](#7-outputs--المخرجات)
8. [Likert scales — مقاييس ليكرت](#8-likert-scales--مقاييس-ليكرت)
9. [Low-level module API](#9-low-level-module-api)
10. [Charts — الرسوم البيانية](#10-charts--الرسوم-البيانية)
11. [MATLAB color palettes — الألوان](#11-matlab-color-palettes--الألوان)
12. [AI / LLM integration — الذكاء الاصطناعي](#12-ai--llm-integration--الذكاء-الاصطناعي)
13. [Streamlit web app — واجهة الويب](#13-streamlit-web-app--واجهة-الويب)
14. [Demo & tests — المثال والاختبارات](#14-demo--tests--المثال-والاختبارات)
15. [Methodological & ethical notes — ملاحظات منهجية](#15-methodological--ethical-notes--ملاحظات-منهجية)

---

## 1. Features — المزايا

| الوحدة Module | ما تقدمه What you get |
|---|---|
| **التحليل الوصفي** Descriptive | sample overview, frequencies & percentages, mean, SD, median, mode, min/max, item ranking, automatic Arabic agreement-level interpretation |
| **مقاييس Likert** | 3/5/7-point Arabic presets, custom mappings, Arabic-text → numeric conversion, diverging stacked bar chart |
| **الثبات** Reliability | Cronbach's alpha, McDonald's omega (ω), corrected item-total correlation, alpha-if-item-deleted, Composite Reliability (CR), AVE |
| **الصدق** Validity | item-dimension correlations, KMO, Bartlett's sphericity, EFA (varimax, Kaiser criterion), Fornell–Larcker, HTMT |
| **الاختبارات الاستدلالية** Inferential | t-test / Mann-Whitney, ANOVA / Kruskal-Wallis (automatic normality-based selection), effect sizes (Cohen's d, η², ε², rank-biserial), chi-square + Cramér's V, Pearson/Spearman/Kendall, OLS regression |
| **جودة البيانات** Data quality | straightlining, low variance, missingness, Mahalanobis multivariate outliers, weak/duplicate open answers — review report only, **no automatic deletion** |
| **الإجابات المفتوحة** Open text | Arabic normalization, keyword extraction, lexicon-based sentiment (with negation), representative quotes, auto Arabic summary paragraph, optional LLM hook |
| **المخرجات** Outputs | full RTL Arabic Word report, styled RTL Excel workbook, raw JSON, PNG charts (Parula palette) |
| **الواجهة** UI | Streamlit web application |

## 2. Installation — التثبيت

```bash
git clone https://github.com/merwanroudane/surveyarabic
cd surveyarabic

pip install -e .          # core library
pip install -e .[app]     # + streamlit & plotly (web interface)
pip install -e .[ai]      # + anthropic SDK (optional LLM analysis)
```

**Core dependencies** (installed automatically): `pandas`, `numpy`, `scipy`,
`statsmodels`, `factor_analyzer`, `matplotlib`, `openpyxl`, `python-docx`,
`arabic-reshaper`, `python-bidi`.

> Compatible with **pandas 3.x** (new `str` dtype) and **scikit-learn 1.8**
> (a built-in shim fixes the `factor_analyzer` ↔ sklearn incompatibility).

## 3. Input data format — صيغة البيانات

One row per respondent, one column per question. Accepted files: `.xlsx`,
`.xls`, `.xlsm`, `.csv` (UTF-8 or Windows-1256) — or pass a ready
`pandas.DataFrame` directly.

| الجنس | العمر | Q1 | Q2 | … | Q10 | ملاحظات |
|---|---|---|---|---|---|---|
| ذكر | 25-35 | موافق | موافق بشدة | … | محايد | الخدمة ممتازة… |
| أنثى | 36-45 | محايد | موافق | … | موافق | أعاني من بطء… |

Likert answers may be **Arabic text** (`موافق بشدة`, …) or **numbers**
(1–5); both are handled. Text matching is normalization-tolerant
(`غير موافق جداً` ≡ `غير موافق جدا`, hamza/ta-marbuta variants unified).

## 4. Quick start — البداية السريعة

```python
from arabic_survey_analyzer import SurveyAnalyzer

analyzer = SurveyAnalyzer("survey.xlsx",
                          title="تقرير تحليل استبيان جودة الخدمة",
                          researcher="د. مروان رودان")

analyzer.define_likert_scale("5-point")

analyzer.set_dimensions({
    "جودة الخدمة": ["Q1", "Q2", "Q3", "Q4"],
    "الرضا":       ["Q5", "Q6", "Q7"],
    "الثقة":       ["Q8", "Q9", "Q10"],
})

analyzer.set_demographics(["الجنس", "العمر", "المستوى التعليمي"])

analyzer.run_all(group_vars=["الجنس"],
                 text_cols=["ملاحظات"],
                 regression_dependent="الرضا")

paths = analyzer.export_all("output")
# output/تقرير_الاستبيان.docx  +  نتائج_الاستبيان.xlsx  +  results.json  +  charts/*.png
```

---

## 5. `SurveyAnalyzer` — full API reference

```python
from arabic_survey_analyzer import SurveyAnalyzer
```

### 5.1 Constructor

```python
SurveyAnalyzer(source, sheet_name=0, title="تقرير تحليل الاستبيان", researcher="")
```

| Parameter | Type | Default | Description |
|---|---|---|---|
| `source` | `str` path or `DataFrame` | — | `.xlsx`/`.xls`/`.csv` file path, or a pandas DataFrame |
| `sheet_name` | `int`/`str` | `0` | Excel sheet to read |
| `title` | `str` | تقرير تحليل الاستبيان | Title shown on the Word report cover |
| `researcher` | `str` | `""` | Researcher name shown on the cover |

On load: column names stripped, fully-empty rows/columns dropped, text
cells trimmed, empty strings → missing.

### 5.2 Configuration methods

All configuration methods return `self`, so they can be chained.

#### `define_likert_scale(scale_type="5-point", mapping=None)`

| Parameter | Type | Description |
|---|---|---|
| `scale_type` | `str` | `"3-point"`, `"5-point"` or `"7-point"` — sets scale bounds and default Arabic label presets |
| `mapping` | `dict` or `None` | optional custom `{label: number}` map; labels are Arabic-normalized before matching |

```python
# preset
analyzer.define_likert_scale("5-point")

# fully custom labels
analyzer.define_likert_scale("5-point", mapping={
    "أرفض بشدة": 1, "أرفض": 2, "لا أدري": 3, "أوافق": 4, "أوافق بشدة": 5,
})
```

#### `set_dimensions(dimensions)`

```python
analyzer.set_dimensions({
    "جودة الخدمة": ["Q1", "Q2", "Q3", "Q4"],
    "الرضا":       ["Q5", "Q6", "Q7"],
})
```

`{dimension_name: [item_columns]}`. Raises `KeyError` listing any column
that does not exist in the data.

#### `set_demographics(columns)`

```python
analyzer.set_demographics(["الجنس", "العمر", "المستوى التعليمي", "سنوات الخبرة"])
```

Columns to describe with frequency tables + pie/bar charts. Unknown
columns are silently skipped.

#### `set_text_columns(columns)`

```python
analyzer.set_text_columns(["ملاحظات", "اقتراحات"])
```

Open-ended question columns for qualitative analysis.

### 5.3 Analysis methods

Each `run_*` method **returns its result table(s)** and also stores them in
`analyzer.results` / `analyzer.tables` (see §6).

#### `run_descriptive()`

```python
items_table, dims_table = analyzer.run_descriptive()
```

Produces: sample overview, demographic frequency tables, per-item
statistics (count, mean, SD, median, mode, min, max, rank, agreement
level), per-dimension statistics, Likert category percentages, and the
auto-written Arabic paragraph `analyzer.paragraphs["dimensions"]`.

#### `run_reliability()`

```python
rel_table = analyzer.run_reliability()
```

Per dimension + overall: Cronbach's α, McDonald's ω, CR, AVE, Arabic
interpretation. Item-total tables per dimension are stored in
`analyzer.results["reliability"]["item_total"]`.

#### `run_validity(n_factors=None)`

| Parameter | Type | Description |
|---|---|---|
| `n_factors` | `int` or `None` | number of EFA factors; `None` → Kaiser criterion (eigenvalues > 1) |

```python
val = analyzer.run_validity()        # dict, see §6
val = analyzer.run_validity(n_factors=3)
```

Produces: item-dimension correlations, KMO + Bartlett table, EFA loadings
+ explained variance + eigenvalues (scree), Fornell–Larcker matrix, HTMT
matrix. If EFA fails (e.g. singular matrix) the error string is stored in
`val["efa_error"]` and the rest still completes.

#### `run_group_tests(group_var, force=None)`

| Parameter | Type | Description |
|---|---|---|
| `group_var` | `str` | grouping column (e.g. `"الجنس"`) |
| `force` | `None` / `"parametric"` / `"nonparametric"` | override the automatic Shapiro-Wilk-based test choice |

```python
t = analyzer.run_group_tests("الجنس")                       # auto choice
t = analyzer.run_group_tests("العمر", force="nonparametric") # always KW/MW
```

Test selection logic:

| Groups | Normal (Shapiro p ≥ .05 in all groups) | Not normal |
|---|---|---|
| 2 | Welch t-test + Cohen's d | Mann-Whitney U + rank-biserial r |
| 3+ | one-way ANOVA + η² | Kruskal-Wallis + ε² |

Can be called repeatedly with different `group_var`s; each adds a table
and an Arabic paragraph.

#### `run_correlations(method="pearson")`

```python
r = analyzer.run_correlations()              # Pearson
r = analyzer.run_correlations("spearman")    # or "kendall"
```

Correlation + p-value matrices between dimension scores
(`analyzer.results["correlation"]["r"]` / `["p"]`).

#### `run_regression(dependent, predictors=None)`

| Parameter | Type | Description |
|---|---|---|
| `dependent` | `str` | dimension name used as the outcome |
| `predictors` | `list[str]` or `None` | predictor dimensions; `None` → all other dimensions |

```python
coef, summary = analyzer.run_regression("الرضا")
coef, summary = analyzer.run_regression("الرضا", predictors=["جودة الخدمة"])
```

OLS on respondent-level dimension scores; returns the coefficient table
(B, SE, t, p, significance) and the model summary (R², adj-R², F, p).

#### `run_data_quality()`

```python
summary = analyzer.run_data_quality()
```

Flags per respondent: straightlining (≥ 90 % identical answers), low
variance (SD < 0.5), high missingness (> 20 %), Mahalanobis outliers
(χ², p < .001), weak/duplicate open answers. Detailed per-respondent
table in `analyzer.results["quality"]["detail"]`. **Nothing is deleted.**

#### `run_text_analysis(columns=None, llm_callable=None)`
*(alias: `run_ai_open_text_analysis` — proposal-compatible name)*

| Parameter | Type | Description |
|---|---|---|
| `columns` | `list[str]` or `None` | open-text columns; `None` → those from `set_text_columns` |
| `llm_callable` | `f(prompt)->str` or `None` | optional LLM for deep thematic analysis (§12) |

```python
out = analyzer.run_text_analysis(["ملاحظات"])
out["ملاحظات"]["keywords"]            # DataFrame: keyword / count / %
out["ملاحظات"]["sentiment_summary"]   # DataFrame: إيجابي/محايد/سلبي + %
out["ملاحظات"]["quotes"]              # list of representative quotes
out["ملاحظات"]["paragraph"]           # auto Arabic summary paragraph
```

#### `run_all(group_vars=None, text_cols=None, regression_dependent=None)`

Runs the full pipeline in the right order: descriptive → data quality →
reliability → validity → group tests (each var) → correlations →
regression (optional) → text analysis (optional).

```python
analyzer.run_all(group_vars=["الجنس", "المستوى التعليمي"],
                 text_cols=["ملاحظات"],
                 regression_dependent="الرضا")
```

### 5.4 Export methods

#### `export_charts(out_dir="charts")`

Renders every chart available for the analyses already run; returns
`{section: [png paths]}` and caches it in `analyzer.charts`.

#### `export_excel(path="نتائج_الاستبيان.xlsx")`

All result tables → one styled workbook (navy headers, zebra rows,
borders, auto column widths, RTL sheet view, frozen header row).

#### `export_report(path="تقرير_الاستبيان.docx", charts_dir="charts")`

Full Arabic academic Word report (see §7.1). Charts are rendered first if
not already cached.

#### `export_json(path="results.json")`

Entire `results` dict serialized to UTF-8 JSON (DataFrames → record
lists) — your machine-readable audit trail.

#### `export_all(out_dir="output")`

Charts + Excel + Word + JSON into one folder. Returns
`{"excel": ..., "word": ..., "json": ..., "charts": ...}` paths.

### 5.5 Attributes

| Attribute | Type | Content |
|---|---|---|
| `analyzer.df` | `DataFrame` | cleaned raw data |
| `analyzer.num_df` | `DataFrame` | data with Likert items converted to numbers |
| `analyzer.scores` | `DataFrame` | respondent-level mean score per dimension |
| `analyzer.items` | `list[str]` | all item columns (property) |
| `analyzer.results` | `dict` | every analysis result (§6) |
| `analyzer.tables` | `dict[str, DataFrame]` | flat table registry → Excel sheets |
| `analyzer.paragraphs` | `dict` | auto-generated Arabic narrative |
| `analyzer.charts` | `dict` | section → list of PNG paths |

---

## 6. Results objects — بنية النتائج

`analyzer.results` keys after `run_all`:

```text
results
├── descriptive
│   ├── overview          DataFrame  — participants / variables / missing
│   ├── demographics      {var: DataFrame}  — frequencies + %
│   ├── items             DataFrame  — per-item stats + rank + level
│   ├── dimensions        DataFrame  — per-dimension stats + rank + level
│   └── percentages       DataFrame  — % per Likert category per item
├── quality
│   ├── detail            DataFrame  — per-respondent flags
│   └── summary           DataFrame  — counts + % per indicator
├── reliability
│   ├── summary           DataFrame  — α, ω, CR, AVE + interpretation
│   └── item_total        {dim: DataFrame}  — r(item, total), α-if-deleted
├── validity
│   ├── item_dimension    DataFrame  — item ↔ dimension r + p
│   ├── kmo_bartlett      DataFrame
│   ├── efa_loadings      DataFrame  — loadings + communalities
│   ├── efa_variance      DataFrame  — eigenvalue / % / cumulative %
│   ├── eigenvalues       list       — for the scree plot
│   ├── fornell_larcker   DataFrame  — √AVE diagonal vs correlations
│   └── htmt              DataFrame
├── group_tests           {group_var: DataFrame}  — test, stat, p, effect size
├── correlation           {"r": DataFrame, "p": DataFrame}
├── regression            {"coefficients": DataFrame, "summary": DataFrame}
└── text                  {column: {keywords, sentiment_detail,
                                    sentiment_summary, quotes,
                                    paragraph, llm_analysis}}
```

---

## 7. Outputs — المخرجات

### 7.1 Word report (RTL) — التقرير العربي

True right-to-left layout (OOXML `<w:bidi/>` paragraphs, `<w:bidiVisual/>`
tables), Traditional Arabic body font, navy-styled tables, embedded
charts. Sections:

1. صفحة عنوان — cover page
2. مقدمة — introduction
3. وصف العينة — sample description (+ demographic charts)
4. جودة البيانات — data-quality summary
5. التحليل الوصفي للفقرات (+ Likert & means charts)
6. تحليل المحاور (+ bar & radar charts) مع فقرة نتائج آلية
7. الثبات (+ α/ω chart) مع فقرة تفسيرية
8. الصدق: KMO/بارتليت، EFA، فورنيل-لاركر، HTMT (+ scree)
9. الاختبارات الاستدلالية (+ boxplots) مع فقرات تفسيرية
10. الارتباط (+ heatmap) والانحدار
11. تحليل الإجابات المفتوحة (+ keywords & sentiment charts، اقتباسات)
12. ملخص النتائج والتوصيات — auto-generated recommendations

### 7.2 Excel workbook — ملف الجداول

One sheet per table (~26 sheets for the full pipeline): نظرة عامة، توزيع
كل متغير ديموغرافي، إحصاءات الفقرات/المحاور، جودة البيانات (ملخص +
تفصيلي)، الثبات، ارتباط الفقرات لكل محور، صدق الاتساق الداخلي، KMO
وبارتليت، التشبعات العاملية، التباين المفسر، فورنيل-لاركر، HTMT، الفروق
حسب كل متغير، مصفوفة الارتباط، الانحدار، الكلمات المفتاحية والمشاعر لكل
سؤال مفتوح.

### 7.3 JSON — سجل التدقيق

Everything in `results` as UTF-8 JSON for reproducibility, audit, or
downstream apps.

---

## 8. Likert scales — مقاييس ليكرت

### Presets — الإعدادات الجاهزة

| `scale_type` | Labels (value) |
|---|---|
| `"3-point"` | غير موافق (1) محايد (2) موافق (3) |
| `"5-point"` | غير موافق بشدة/جداً (1) غير موافق (2) محايد (3) موافق (4) موافق بشدة/جداً (5) |
| `"7-point"` | غير موافق بشدة (1) … محايد (4) … موافق بشدة (7) |

### Agreement levels — مستويات الموافقة

Equal-width thirds of the scale range. For a 5-point scale:

| Mean range | Level |
|---|---|
| 1.00 – 2.33 | منخفض |
| 2.34 – 3.67 | متوسط |
| 3.68 – 5.00 | مرتفع |

```python
from arabic_survey_analyzer import agreement_level
agreement_level(4.12)                          # 'مرتفع'   (5-point default)
agreement_level(4.12, scale_min=1, scale_max=7) # 7-point scale
```

---

## 9. Low-level module API

Every stage is usable standalone with plain DataFrames.

### 9.1 Reading data

```python
from arabic_survey_analyzer import read_survey
df = read_survey("survey.xlsx")            # or .csv (UTF-8 / Windows-1256)
df = read_survey(existing_dataframe)       # pass-through + cleaning
```

### 9.2 Likert conversion

```python
from arabic_survey_analyzer.likert import build_mapping, to_numeric

mapping = build_mapping("5-point")                       # preset
mapping = build_mapping(mapping={"أوافق": 4, ...})       # custom
num_df  = to_numeric(df, items=["Q1", "Q2"], mapping=mapping)
```

### 9.3 Descriptives

```python
from arabic_survey_analyzer.descriptive import (
    sample_overview, frequency_table, demographics_tables,
    item_statistics, dimension_statistics, dimension_scores,
    likert_percentages)

item_statistics(num_df, ["Q1", "Q2"], scale_min=1, scale_max=5)
dimension_statistics(num_df, {"محور أ": ["Q1", "Q2"]})
scores = dimension_scores(num_df, dimensions)   # n × dims, for tests/SEM
```

### 9.4 Reliability

```python
from arabic_survey_analyzer import cronbach_alpha, mcdonald_omega
from arabic_survey_analyzer.reliability import (
    item_total_statistics, composite_reliability_ave, reliability_table)

cronbach_alpha(num_df[["Q1", "Q2", "Q3"]])      # float
mcdonald_omega(num_df[["Q1", "Q2", "Q3"]])      # float (1-factor ω total)
item_total_statistics(num_df[["Q1", "Q2", "Q3"]])  # DataFrame
cr, ave = composite_reliability_ave(num_df[["Q1", "Q2", "Q3"]])
reliability_table(num_df, dimensions)           # full Arabic table
```

### 9.5 Validity

```python
from arabic_survey_analyzer.validity import (
    item_dimension_correlations, kmo_bartlett, efa,
    fornell_larcker, htmt_matrix)

table, kmo_value, bartlett_p = kmo_bartlett(num_df, items)
loadings, variance, eigenvalues = efa(num_df, items)            # Kaiser
loadings, variance, eigenvalues = efa(num_df, items, n_factors=3,
                                      rotation="varimax")       # fixed k
fornell_larcker(num_df, dimensions)
htmt_matrix(num_df, dimensions)
```

### 9.6 Inferential tests

```python
from arabic_survey_analyzer.inferential import (
    compare_groups, chi_square_table, correlation_matrix, linear_regression)

compare_groups(num_df, df["الجنس"], scores)                  # auto test
compare_groups(num_df, df["العمر"], scores, force="parametric")
res, crosstab = chi_square_table(df, "الجنس", "المستوى التعليمي")
r_mat, p_mat  = correlation_matrix(scores, method="spearman")
coef, summary = linear_regression(scores, "الرضا", ["جودة الخدمة", "الثقة"])
```

### 9.7 Data quality

```python
from arabic_survey_analyzer.data_quality import (
    straightlining, low_variance, missing_per_respondent,
    mahalanobis_outliers, weak_text_answers, quality_report)

detail, summary = quality_report(df, num_df, items, text_cols=["ملاحظات"])
straightlining(num_df, items, threshold=0.9)
mahalanobis_outliers(num_df, items, alpha=0.001)
```

### 9.8 Text analysis

```python
from arabic_survey_analyzer.text_analysis import (
    clean_answers, keyword_frequencies, sentiment_score, sentiment_analysis,
    representative_quotes, analyze_open_text)
from arabic_survey_analyzer.textutils import normalize_ar, ar

keyword_frequencies(df["ملاحظات"], top_n=20)
sentiment_score("الخدمة ممتازة والموظفون متعاونون")    # 1.0
detail, summary = sentiment_analysis(df["ملاحظات"])
normalize_ar("إستبيانٌ")     # 'استبيان' (hamza/diacritics unified)
ar("جودة الخدمة")            # reshaped+bidi string for matplotlib labels
```

---

## 10. Charts — الرسوم البيانية

All charts: Arabic-safe text (arabic-reshaper + python-bidi, Tahoma),
150 dpi PNG, Parula palette by default. Each function saves a PNG and
returns its path.

```python
from arabic_survey_analyzer import visualization as viz
```

| Function | Chart |
|---|---|
| `viz.likert_diverging_chart(pct_df, out_dir)` | diverging stacked bars (Heiberger–Robbins) per item |
| `viz.item_means_chart(item_table, out_dir, scale_max=5)` | horizontal item means + SD error bars |
| `viz.dimension_means_chart(dim_table, out_dir, scale_max=5)` | dimension means + SD |
| `viz.dimensions_radar_chart(dim_table, out_dir, scale_max=5)` | radar profile (needs ≥ 3 dims) |
| `viz.correlation_heatmap(r_mat, out_dir, colorscale="Parula")` | annotated heatmap |
| `viz.demographic_chart(freq_table, var_name, out_dir)` | pie (≤ 5 categories) or bars |
| `viz.group_boxplot(scores, df["الجنس"], "الرضا", out_dir)` | boxplots by group |
| `viz.scree_plot(eigenvalues, out_dir)` | scree + Kaiser line |
| `viz.reliability_chart(rel_table, out_dir)` | α vs ω bars + 0.70 threshold |
| `viz.keywords_chart(kw_df, col, out_dir)` | top-15 keyword bars |
| `viz.sentiment_chart(sent_summary, col, out_dir)` | sentiment pie |
| `viz.interactive_dimension_chart(dim_table)` | **plotly** interactive bar (returns Figure) |

```python
path = viz.dimension_means_chart(dims_table, "charts", scale_max=5)
fig  = viz.interactive_dimension_chart(dims_table, colorscale="Parula")
fig.show()
```

---

## 11. MATLAB color palettes — الألوان

`parula_colors(n)` reproduces MATLAB R2014b Parula from the official 64
RGB stops; Parula is the default everywhere.

```python
from arabic_survey_analyzer import (parula_colors, matlab_jet_colors,
                                    turbo_colors, bluered_colors,
                                    sinha_colors, resolve_colorscale)

parula_colors(8)            # ['#352a87', '#2058b0', ...]  8 hex colours
matlab_jet_colors(16)       # classic MATLAB Jet
turbo_colors(16)            # Google Turbo
bluered_colors(16)          # blue-white-red diverging
sinha_colors(16)            # navy-teal-green-gold-red ramp

resolve_colorscale("Parula")        # plotly colorscale [[0.0,'#352a87'],...]
from arabic_survey_analyzer.colors import get_cmap
cmap = get_cmap("Parula")           # matplotlib colormap object
```

---

## 12. AI / LLM integration — الذكاء الاصطناعي

The library never *requires* an API. Any function `f(prompt: str) -> str`
works as `llm_callable`; two convenience builders are provided.

### Local model (full privacy — recommended for sensitive data)

```python
from arabic_survey_analyzer.ai_analysis import make_openai_compatible_callable

llm = make_openai_compatible_callable("http://localhost:11434/v1",
                                      model="qwen2.5")        # Ollama
analyzer.run_text_analysis(["ملاحظات"], llm_callable=llm)
```

### Anthropic API

```python
from arabic_survey_analyzer.ai_analysis import make_anthropic_callable

llm = make_anthropic_callable(model="claude-sonnet-4-6")  # uses ANTHROPIC_API_KEY
analyzer.run_text_analysis(["ملاحظات"], llm_callable=llm)
```

### Custom function

```python
def my_llm(prompt: str) -> str:
    ...  # anything: requests.post to your server, a pipeline, etc.
    return generated_text

analyzer.run_text_analysis(["ملاحظات"], llm_callable=my_llm)
```

The LLM output appears under "تحليل الذكاء الاصطناعي" in the Word report
and in `results["text"][col]["llm_analysis"]`. LLM failures never break
the pipeline — the error message is stored instead.

---

## 13. Streamlit web app — واجهة الويب

```bash
streamlit run arabic_survey_analyzer/streamlit_app.py
```

Workflow: upload file → preview → pick Likert scale, demographics, open
questions and dimensions in the sidebar → run → browse 6 result tabs
(الوصفي / المحاور / الثبات والصدق / الفروق / الإجابات المفتوحة / جودة
البيانات) → download Word / Excel / JSON.

---

## 14. Demo & tests — المثال والاختبارات

```bash
python examples/generate_sample_data.py   # synthetic Arabic survey (n=220)
python examples/run_demo.py               # full pipeline -> examples/output/
python -m pytest tests -q                 # 8 unit tests
```

The synthetic data has person-level latent traits per dimension, so the
demo produces realistic psychometrics (α ≈ 0.81–0.84, KMO ≈ 0.74).

---

## 15. Methodological & ethical notes — ملاحظات منهجية

- مخرجات الذكاء الاصطناعي **احتمالية** وليست حكماً نهائياً، ولا تعوض الحكم العلمي للباحث.
- لا يُحذف أي مشارك آلياً؛ وحدة جودة البيانات تنتج **تقرير مراجعة** فقط.
- استخدم نموذجاً محلياً عند تحليل بيانات حساسة، ولا تُرسل بيانات شخصية لأي خدمة خارجية دون موافقة.
- كل خطوة تحليلية موثقة في `results.json` (سجل تدقيق Audit trail).
- ω يحسب من نموذج عامل واحد (minres)؛ عند فشل التقدير يستخدم احتياطي PCA.
- اختيار الاختبار المعلمي/اللامعلمي يعتمد على Shapiro-Wilk داخل كل مجموعة (عينة ≤ 500)، ويمكن فرضه يدوياً عبر `force=`.

## License

MIT © Dr Merwan Roudane
