Metadata-Version: 2.4
Name: lumi-language-id-2
Version: 2.0.0
Summary: For when you want fastText language identification, but you also want to believe the answers
Project-URL: Repository, https://github.com/whr778/lumi_language_id
Author-email: Robyn Speer <rspeer@luminoso.com>
Maintainer-email: William Roe <whr778@gmail.com>
License-Expression: MIT
License-File: LICENSE.txt
Keywords: fasttext,language-detection,language-identification,nlp
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Text Processing :: Linguistic
Requires-Python: >=3.9
Requires-Dist: fasttext-numpy2
Requires-Dist: ftfy
Requires-Dist: langcodes>=2
Requires-Dist: numpy
Provides-Extra: train
Requires-Dist: scikit-learn; extra == 'train'
Description-Content-Type: text/markdown

# `lumi_language_id`

Utilities for reliable-enough language detection.

This is a maintained fork of the original (archived) `lumi-language-id`,
updated to work with numpy 2.x. It is published on PyPI as
**`lumi-language-id-2`**, but the import name stays `lumi_language_id`:

    pip install lumi-language-id-2   # or: uv add lumi-language-id-2

This package wraps fastText's "lid.176" language-detection model with another
classifier, which is trained to produce better probability estimates.  It also
applies text cleaning, so that the text it detects is unaffected by
punctuation, digits, or emoji.

Example:

    >>> from lumi_language_id.tuned import TunedLanguageIdentifier
    >>> lid = TunedLanguageIdentifier.load()
    >>> lang, _prob = lid.detect_language("these are words")
    >>> lang
    'en'
    
    >>> lang, _prob = lid.detect_language("aquí hay algunas palabras")
    >>> lang
    'es'
