Metadata-Version: 2.4
Name: ovos-lang-detect-plugin-linguonnx
Version: 0.1.0a1
Summary: OVOS language detection plugin using linguonnx (GlotLID/ONNX)
Author-email: jarbasAi <jarbasai@mailfence.com>
License: Apache-2.0
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: linguonnx
Requires-Dist: ovos-plugin-manager<3.0.0,>=2.1.0
Provides-Extra: test
Requires-Dist: pytest; extra == "test"
Dynamic: license-file

# ovos-lang-detect-plugin-linguonnx

An OVOS language-detection plugin. It wraps
[`linguonnx`](https://github.com/TigreGotico/linguonnx), which runs the
[`TigreGotico/glotlid-onnx`](https://huggingface.co/TigreGotico/glotlid-onnx)
export of GlotLID on `onnxruntime`, CPU-only, no torch.

## Install

```bash
pip install ovos-lang-detect-plugin-linguonnx
```

The model downloads from HuggingFace on first use and is cached under
`~/.cache/linguonnx/models/<model_id>/`. Constructing the plugin does not
trigger this download; it happens on the first call to `detect`,
`detect_probs`, or `available_languages`.

## Configuration

```json
{
  "language_detection": {
    "module": "ovos-lang-detect-plugin-linguonnx",
    "ovos-lang-detect-plugin-linguonnx": {
      "model": "glotlid-int8",
      "collapse_varieties": true,
      "min_confidence": 0.0,
      "lang": "en-US"
    }
  }
}
```

- `model`: `"glotlid-int8"` (default, quantized, 419 MB) or `"glotlid"`
  (fp32, 1.68 GB). Both cover the same 2102 GlotLID labels; the int8 model
  has no measured accuracy loss, so there is no reason to prefer the fp32
  one unless you need it for some other purpose.
- `collapse_varieties`: see below.
- `min_confidence`: if the model's top confidence is below this, `detect()`
  returns `self.config.get("lang", "en-US")` instead of the detected tag.
  Default `0.0`, meaning no fallback ever triggers.
- `lang`: the fallback tag used when `min_confidence` isn't met. Default
  `"en-US"`.

## Language varieties

GlotLID identifies individual language varieties, not macrolanguages.
Casual Arabic text comes back as e.g. `ajp-Arab` (South Levantine) rather
than `ar`, and some Chinese text comes back as `yue-Hani` (Cantonese)
rather than `zh`.

`linguonnx` itself defaults to reporting these varieties as-is, since that
fine-grained answer is useful on its own (free dialect identification).
This plugin flips the default to collapse varieties onto their
macrolanguage (`ajp-Arab` -> `ar`), because the two things OVOS actually
does with a detected language tag - picking a TTS voice, picking a
translation target - only know macrolanguages. A caller getting
`ajp-Arab` back from a pipeline that only ships `ar` voices would read it
as unsupported.

Set `collapse_varieties: false` in the plugin config to get `linguonnx`'s
native per-variety fidelity instead.

## Credits

- [linguonnx](https://github.com/TigreGotico/linguonnx) - the detection engine
  this plugin wraps.
- [TigreGotico/glotlid-onnx](https://huggingface.co/TigreGotico/glotlid-onnx) -
  the ONNX export of GlotLID that `linguonnx` runs.
