Metadata-Version: 2.4
Name: re-spell-engine
Version: 1.1.0
Summary: A dual-channel ML-based spelling correction engine for Movies and TV search, trained on TMDB vocabulary.
Author-email: Developer <developer@example.com>
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.8
Description-Content-Type: text/markdown
Requires-Dist: scikit-learn>=1.2.0
Requires-Dist: rapidfuzz>=3.0.0
Requires-Dist: numpy>=1.24.0
Requires-Dist: pandas>=2.0.0
Requires-Dist: requests>=2.31.0
Requires-Dist: python-dotenv>=1.0.0

# Re-Spell Engine

<p align="center">
  <img src="../app/respell-logo_dark.png" alt="Re-Spell Logo" width="280">
</p>

A production-ready, high-performance spelling correction engine for Movie and TV Show searches, trained on TMDB content.

This engine implements a **Dual-Channel Retrieval Model** (character n-gram TF-IDF on full titles and consonant-only skeletons) backed by **Damerau-Levenshtein distance alignment** and custom **phonetic vowel-substitution heuristics** to resolve complex structural and phonetic typos (e.g., `betmen`, `batmen`, `batmn` $\rightarrow$ `Batman`, and `bahubli` $\rightarrow$ `Bāhubali`).

---

## Installation

Install the package directly from PyPI:

```bash
pip install re-spell-engine
```

---

## Using as a Python Library

You can import and integrate the spelling engine directly into other Python applications. The package includes a pre-trained default model binary of over 108,000+ Movies and TV Shows.

```python
from respell_engine import SpellingCorrector

# 1. Initialize the corrector
corrector = SpellingCorrector()

# 2. Load the default pre-trained model shipped with the package
# (You can optionally pass a custom path to corrector.load("/path/to/custom/model.pkl"))
corrector.load()

# 3. Correct a misspelled query
query = "betmen"
result = corrector.correct(query)

if result["is_corrected"]:
    print(f"Original: {result['original_query']}")
    print(f"Corrected Suggestion: {result['corrected_query']}")
    print(f"Confidence: {result['confidence']:.4f}")
    print(f"Autocorrect Recommended: {result.get('is_autocorrect', False)}")
else:
    print("Spelling is correct or no confident correction was found.")
```

### Correction Response Schema
The `.correct()` method returns a dictionary with the following fields:
- `original_query` *(str)*: The input query string.
- `corrected_query` *(str)*: The clean base corrected title (e.g. `Bāhubali` instead of `Bāhubali: The Epic`) to maximize the downstream search scope.
- `did_you_mean` *(str | None)*: Suggestion string if correction is confident; otherwise `None`.
- `is_corrected` *(bool)*: `True` if a spelling correction was determined.
- `is_autocorrect` *(bool)*: `True` if correction confidence is extremely high (e.g., $\ge 0.83$ or edit distance $\le 1.0$) recommending immediate query substitution.
- `confidence` *(float)*: Score representing the quality of match (bounded between `0.0` and `1.0`).
- `candidate_details` *(dict | None)*: Underlying metadata of matched candidate (id, media_type, distance, raw score, etc.).

### Web App Integration Example (Streamlit)
For high-performance in-process usage inside web frontends, cache the spelling corrector instance to prevent loading the model on every page refresh:

```python
import streamlit as st
from respell_engine import SpellingCorrector

@st.cache_resource
def get_corrector():
    corrector = SpellingCorrector()
    corrector.load()  # Automatically loads package data corrector.pkl
    return corrector

corrector = get_corrector()
query = st.text_input("Search")

if query:
    result = corrector.correct(query)
    if result["is_corrected"]:
        st.write(f"Showing results for: {result['corrected_query']}")
```

---


## Web API Integration

Start the web server locally using `uvicorn` (runs CPU-bound NLP logic on a worker thread pool for production concurrency):

```bash
uvicorn respell_engine.main:app --host 127.0.0.1 --port 8000
```

### Endpoints

#### 1. Search with Spelling Correction
- **Route**: `GET /api/search`
- **Parameters**:
  - `query` *(str, required)*: The search input query.
  - `autocorrect` *(bool, optional, default=true)*: If set to false, disables correction logic and returns query literally.
- **Example request**: `GET /api/search?query=betmen`
- **Example Response**:
  ```json
  {
      "original_query": "betmen",
      "corrected_query": "Batman",
      "did_you_mean": "Batman",
      "is_corrected": true,
      "is_autocorrect": true,
      "confidence": 0.9035,
      "candidate_details": {
          "movie_id": 2287,
          "display_title": "Batman",
          "media_type": "tv",
          "distance": 1.0,
          "norm_dist": 0.1666,
          "cosine_sim": 0.0787,
          "score": 0.9035
      }
  }
  ```

#### 2. Get Engine Status
- **Route**: `GET /api/status`

#### 3. Trigger Retraining
- **Route**: `POST /api/train`

---

## Ingestion & Training Workflow

If you want to train on a fresh dataset from TMDB:

### 1. Set environment variables
Create a `.env` file at the root of your project directory containing your TMDB Read Access Token (Bearer Token):
```env
TMDB_API_TOKEN=your_bearer_token_here
```

### 2. Ingest Content
```bash
python3 -m respell_engine.scripts.fetch_data
```

### 3. Re-train Model
```bash
python3 -m respell_engine.scripts.train
```

### 4. Verify Accuracy
```bash
python3 -m respell_engine.scripts.generate_evaluation_log
```

---

## Changelog

### v1.1.0 (Current Release)
- **Enhanced Accuracy**: Reached **98.5% overall accuracy** on a expanded test suite of **200 synthetic and real-world typos** (added keyboard adjacency errors and double-letter insertions).
- **Consonant Skeleton Improvements**: Added `'y'` to the English vowel list in consonant skeleton generation, resolving query matches like `the bos` -> `The Boys`.
- **Short-Query Adjustments**: Relaxed length boundaries on consonant skeleton matching, allowing 2-3 char abbreviations (like `hs` -> `House`) to correct safely under exact skeleton matches.
- **Space-Omission Fast Match**: Added a fast spaceless dictionary index lookup ($O(1)$) to instantly resolve space-omission errors (e.g. `laworderspecialvictimsunit` -> `Law & Order`).
- **Data Ingestion Expansion**: Added weekly trending discovery and multi-genre crawls (Action, Sci-Fi, Horror, etc.) for both Movies and TV Shows. Expanded original languages to Portuguese, Russian, Arabic, Thai, Turkish, and Polish, growing the unique vocabulary to **108,833 items**.
- **Package Trimming**: Cleaned unused packaging requirements, making the core PyPI package lightweight.
