Metadata-Version: 2.4
Name: ovos-utterance-normalizer
Version: 0.2.5a2
Author-email: jarbasai <jarbasai@mailfence.com>
License-Expression: Apache-2.0
Project-URL: Homepage, https://github.com/OpenVoiceOS/ovos-utterance-normalizer
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Topic :: Text Processing :: Linguistic
Classifier: Programming Language :: Python :: 3.4
Classifier: Programming Language :: Python :: 3.5
Classifier: Programming Language :: Python :: 3.6
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: ovos-utils<1.0.0,>=0.0.38
Requires-Dist: quebra-frases
Requires-Dist: ovos-plugin-manager<3.0.0,>=2.1.0
Requires-Dist: ftfy
Dynamic: license-file

# ovos-utterance-normalizer

`ovos-utterance-normalizer` is an [OVOS](https://github.com/OpenVoiceOS) plugin that
normalizes utterances before intent parsing. It expands contractions, converts spoken
numbers to digits, strips punctuation, and applies other language-specific cleanup so
that intent parsers get a consistent input.

The plugin also fixes text encoding errors with
[python-ftfy](https://github.com/rspeer/python-ftfy).

It is enabled by default in
[mycroft.conf](https://github.com/OpenVoiceOS/ovos-config/blob/8cfb04319516cad38d39203d14f10d6f0f568390/ovos_config/mycroft.conf#L114).

## Install

```bash
pip install ovos-utterance-normalizer
```

## Usage

`ovos-utterance-normalizer` registers as an
[OVOS PHAL](https://github.com/OpenVoiceOS/ovos-plugin-manager) `opm.transformer.text`
plugin, named `ovos-utterance-normalizer`. [ovos-core](https://github.com/OpenVoiceOS/ovos-core)
loads it automatically when it is installed and enabled in the configuration.

For each utterance, the plugin produces up to three variants: the utterance with
contractions expanded, the original utterance, and the normalized utterance. Punctuation
is stripped from each variant, and duplicates are removed while keeping the original order.

Supported configuration options, under the plugin name in `mycroft.conf`:

| Option | Default | Description |
|---|---|---|
| `lang` | `en-us` | Fallback language, used when no language is given in the utterance context |
| `fix_encoding_errors` | `true` | Fix encoding errors with `ftfy` before normalizing |
| `strip_punctuation` | `true` | Strip punctuation from each output variant |

Normalization is language-specific. English, Portuguese, Ukrainian, Catalan, Czech,
Azerbaijani, Russian, and German each have a dedicated normalizer. Other languages fall
back to the generic normalizer.

## Related projects

- [ovos-plugin-manager](https://github.com/OpenVoiceOS/ovos-plugin-manager), which loads
  and runs this plugin as an `opm.transformer.text` transformer.
- [ovos-config](https://github.com/OpenVoiceOS/ovos-config), which ships the default
  `mycroft.conf` that enables this plugin.
- [ovos-core](https://github.com/OpenVoiceOS/ovos-core), the assistant runtime that
  calls transformer plugins before intent parsing.

## License

Apache-2.0
