Metadata-Version: 2.4
Name: usefsufe
Version: 0.1.0
Summary: Learn Neural Networks, Deep Learning and NLP step by step: a visual desktop app plus from-scratch NLTK-style NLP and Keras/PyTorch-style deep learning libraries.
Author: Yousef Sameh
License-Expression: MIT
Keywords: jupyter,colab,education,neural networks,deep learning,nlp,nltk,perceptron,backpropagation,teaching,from scratch,wordnet,lstm,attention
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Education
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Education
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.21
Requires-Dist: matplotlib>=3.5
Provides-Extra: notebook
Requires-Dist: ipywidgets>=7.6; extra == "notebook"
Provides-Extra: dev
Requires-Dist: pytest>=7; extra == "dev"
Requires-Dist: build; extra == "dev"
Requires-Dist: twine; extra == "dev"
Requires-Dist: ipywidgets>=7.6; extra == "dev"
Requires-Dist: nltk; extra == "dev"
Requires-Dist: scikit-learn; extra == "dev"
Dynamic: license-file

# UsefsUFE

**Learn Neural Networks, Deep Learning and NLP step by step.**

UsefsUFE is two things in one package:

1. **A desktop app** where you click *Next ▶* and watch an algorithm work, one step at a
   time. Every step shows a picture and explains in plain English what just happened,
   with the real numbers filled into the formulas.
2. **A Python library written from scratch** in NumPy: a Perceptron, neural networks with
   hand-written backpropagation, and `uf.nlp`, an NLTK-compatible NLP toolkit (tokenizers,
   stemmers, WordNet, POS tagging, chunking, parsing, language models, VADER sentiment,
   word2vec, BLEU and more) with the same names as NLTK.

It needs only NumPy and Matplotlib, and works fully offline: everything it needs is bundled
(Iris, handwritten digits, public-domain books, a treebank sample, WordNet, the VADER lexicon,
80 teaching reviews). Sources and licences: [`data/DATA_LICENSES.md`](src/usefsufe/data/DATA_LICENSES.md).

![Start screen](docs/screenshots/0_home.png)

## Install and run

```bash
pip install usefsufe
usefsufe
```

The first command installs the package and its dependencies. The second opens the app.
You can also start it with `python -m usefsufe`.

**In Google Colab or Jupyter** (no desktop window there), use the notebook player:

```python
!pip install -q usefsufe
from usefsufe.notebook import browse
browse()
```

See [`examples/UsefsUFE_Colab.ipynb`](examples/UsefsUFE_Colab.ipynb) and the full
[usage guide](docs/USAGE.md) for every function, setting and command.

> **Windows:** Tkinter comes with the standard Python installer from python.org.
> If the app says Tkinter is missing, re-run the installer and tick *"tcl/tk and IDLE"*.

## Start screen, Code Reference and Tips

The app opens on a **start screen**: pick a course (Neural Networks, Deep Learning or NLP),
open the **Code Reference**, or read the **Tips & Summaries**. The *Home* button (or `Esc`)
always brings you back.

* **Code Reference** - every function grouped and searchable, each with *when to use it*,
  its signature, a one-line example you can **Run** (plots open in a window) or **Copy**,
  the NLTK / scikit-learn equivalent and a link to the related lesson. The same reference
  works without the app: `uf.menu()`, `uf.menu("pos_tag")`, `uf.menu(search="stem")` or
  `uf.menu(interactive=True)`.
* **Tips & Summaries** - per course, a *when to use what* table and every lesson's key
  points on one page. Inside each lesson, the **Summary** tab shows the same without
  pressing *Next*.

![Code Reference](docs/screenshots/14_code_reference.png)
![Tips](docs/screenshots/15_tips.png)

## The lessons

| Tab | Lesson | What you see |
|---|---|---|
| Neural Networks | Activation functions explorer | Sigmoid (with slope *a*), tanh, ReLU and swish, their slopes and the vanishing gradient |
| Neural Networks | Perceptron, step by step | Every update on Iris with `Δw = η(y − ŷ)x` filled in; why η changes nothing from a zero start; versicolor never converges; the pocket algorithm |
| Neural Networks | XOR: where one neuron fails | The failure, a 4-line proof, the OR/AND hidden-layer fix, and a network that learns it |
| Neural Networks | Three classes: softmax | One-vs-rest and its grey "no answer" areas, softmax probabilities, a confusion matrix |
| Deep Learning | Linear model vs deep network | Two moons: a straight line vs a boundary that bends |
| Deep Learning | The linearity trap | A deep network without activations, proven equal to one layer using its own weights |
| Deep Learning | Vanishing gradients | Gradient size per layer, a 6-layer sigmoid net stuck at log 2 with SGD, ReLU and Adam as fixes, exploding gradients at high learning rates |
| Deep Learning | Overfitting | Loss vs accuracy vs over-confidence on unseen data; early stopping that restores the best weights; L2 |
| Deep Learning | Inside one training step | A 2-2-1 network: forward, loss, backprop, a gradient check, the update |
| Deep Learning | Autograd: how .backward() works | The lab's `x0² + x1³` example on a graph, node by node (gradients 4 and 27), then a neuron |
| NLP | From raw text to numbers | Characters → sentences → cleaning (with contraction expansion) → tokens → stop words → counts → vocabulary |
| NLP | The negation pitfall | How "not good" becomes "good", and three fixes |
| NLP | Mini sentiment classifier | The lab's word-voting classifier, its honest cross-validated score, and where it fails |
| NLP | From words to a neural network | Reviews → bag of words → Perceptron and a small network; 5-fold comparison of negation handling |

Each step shows a picture and **one or two lines** of explanation; **More ▾** opens the full
explanation with every number. The **Summary** tab shows the whole lesson on one page.

![Short text with More](docs/screenshots/16_short_text.png)

Every lesson has settings on the left (learning rate, number of neurons, your own text…).
Change one and the lesson recomputes, staying on the same step. Settings that have no
effect on the current step are greyed out. Every sentence that describes a result ("the loss
has flattened", "they differ") is computed from the actual numbers, never hard-coded.

**Code tab:** every lesson shows the Python code that does the same thing with the library,
filled in with your current settings. Copy it or save it as a `.py` file.

![Code tab](docs/screenshots/10_code_tab.png)

**Controls:** `◀ Back`, `Next ▶`, `Reset`. Keyboard: ← and → to step. *Save picture* exports the current plot as PNG or PDF.

![Linear vs deep](docs/screenshots/4_linear_vs_deep.png)
![Backpropagation](docs/screenshots/6_backprop.png)
![Overfitting](docs/screenshots/8_overfitting.png)
![XOR](docs/screenshots/9_xor.png)

## Use it from Python

```python
import numpy as np
from usefsufe.core import Perceptron, MLP
from usefsufe.core.datasets import load_iris, make_moons, train_test_split
from usefsufe.core import text

# Perceptron on Iris (setosa vs the rest), like the NN lab
X, species, names = load_iris()
y = np.where(species == "Iris-setosa", 1, -1)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.25)
model = Perceptron(eta=0.1, n_iter=10).fit(X_tr, y_tr)
print(model.errors_, model.score(X_te, y_te))

# A deep network on two moons, like the DL lab
X, y = make_moons(400, noise=0.2)
net = MLP([2, 16, 16, 1], activation="relu", lr=0.01, l2=0.001).fit(X, y, epochs=300)
print(net.score(X, y))

# The NLP lab's tools, without downloading anything
print(text.word_tokenize("Battery isn't bad!"))        # ['Battery', 'is', "n't", 'bad', '!']
print(text.clean("Battery isn't bad!", expand=True))   # 'battery is not bad'
print(text.remove_stopwords(["not", "good"]))          # ['good']   <- the pitfall
print(text.remove_stopwords(["not", "good"], keep_negations=True))
```

Watch a training run step by step:

```python
for info in Perceptron(eta=0.1, n_iter=2).fit_steps(X_tr, y_tr):
    if info["update"] != 0:
        print(info["epoch"], info["w_before"], "->", info["w_after"])
```

The NLP toolkit (NLTK names) and the deep learning library (Keras and PyTorch style):

```python
import usefsufe as uf

tokens = uf.nlp.word_tokenize("The battery isn't bad, but the screen is AMAZING!")
uf.nlp.pos_tag(tokens)
uf.nlp.SentimentIntensityAnalyzer().polarity_scores("The battery isn't bad!")
uf.nlp.WordNetLemmatizer().lemmatize("better", "a")          # 'good'

from usefsufe.dl import Sequential, Dense
model = Sequential([Dense(16, activation="relu"), Dense(1, activation="sigmoid")])
model.compile(optimizer="adam", loss="binary_crossentropy", metrics=["accuracy"])
model.fit(X, y, epochs=50)
print(uf.metrics.classification_report(y, model.predict_classes(X)))

uf.menu()            # browse every function with an example
```

Step through a lesson in a Jupyter notebook (pictures and explanations, step by step):

```python
from usefsufe.lessons import get_lesson
get_lesson("linearity_trap").show_steps(epochs=200)
```

## How it is built

```
src/usefsufe/
  core/        the lesson maths: activations, Perceptron, MLP (backprop), softmax, autograd, text basics
  nlp/         NLTK-style NLP from scratch (tokenizers, stemmers, WordNet, taggers, parsers, LMs, VADER...)
  dl/          deep learning from scratch: layers, losses, optimizers, Sequential, callbacks, data
  metrics.py   classification and regression metrics
  lessons/     each lesson is a generator that yields Steps (short text, full text, data to draw)
  catalog.py   the code reference: every function with "when to use it" and a runnable example
  menu.py      uf.menu(): the code reference in a terminal or notebook
  tips.py      "when to use what" tables and lesson summaries
  app/         the Tkinter app: start screen, course pages, Code Reference, Tips
  notebook.py  the lesson player for Jupyter / Colab (ipywidgets)
  data/        bundled datasets and models (see DATA_LICENSES.md)
tools/build_data.py   rebuilds the bundled data (developers only)
```

The app contains no maths. Each lesson computes one step, pauses, and waits for *Next*.
That is why heavy training happens gradually and *Back* is instant.

## Development

```bash
python -m venv .venv
.venv\Scripts\activate            # Windows   (Linux/macOS: source .venv/bin/activate)
pip install -e ".[dev]"
pytest                             # 360 tests: maths vs. the lab results, NLTK / scikit-learn comparisons, gradient checks, every lesson, every code-reference example, the app
usefsufe
```

## License

MIT. See [LICENSE](LICENSE).

Lesson content follows the Neural Networks, Deep Learning and NLP labs at
Université Française d'Égypte (UFE).
