Metadata-Version: 2.5
Name: vn-text
Version: 0.1.0
Summary: Correct Unicode primitives for Vietnamese text: deaccent, fold, normalize, strip_stroke. Standard library only. Ships a cross-language conformance suite shared with the TypeScript port.
Project-URL: Homepage, https://github.com/leeloc1809/vn-toolkit
Project-URL: Repository, https://github.com/leeloc1809/vn-toolkit
Project-URL: Issues, https://github.com/leeloc1809/vn-toolkit/issues
License: MIT
License-File: LICENSE
Keywords: diacritics,i18n,nfc,nfd,normalization,search,slugify,unicode,vietnam,vietnamese
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Natural Language :: Vietnamese
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Text Processing :: Linguistic
Classifier: Typing :: Typed
Requires-Python: >=3.9
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == 'dev'
Description-Content-Type: text/markdown

# vn-text

Correct Unicode primitives for Vietnamese text. **No runtime dependencies** —
standard library only.

```bash
pip install vn-text
```

```python
from vn_text import fold, deaccent, strip_stroke, is_vietnamese

fold('Đặng Minh Anh')     # 'dang minh anh'
fold('Hà Nội, Việt Nam!')  # 'ha noi viet nam'
deaccent('Tiếng Việt')    # 'Tieng Viet'
strip_stroke('Đặng')      # 'Dặng'
is_vietnamese('Nguyễn')   # True
```

## Why

Vietnamese `Đ` (U+0110) has no canonical decomposition, so NFD cannot fold it.
The usual "NFD then strip combining marks" recipe leaves `Đặng Minh` as
`Đang Minh` — and nobody types `Đang` to find `Đặng`, so search quietly returns
too few results. The tempting blanket fix also turns the Icelandic letter `Ð`
(U+00D0) into `D` and corrupts those names.

`strip_stroke` handles U+0110 and U+0111 explicitly, and nothing else.

## Validation

Every function is checked against the shared conformance suite at
[`conformance/vn-text-1.0.0.json`](../../conformance/vn-text-1.0.0.json) — 156
hand-authored cases that the TypeScript port also consumes, so the two
implementations cannot drift apart.

The test file runs with or without pytest, because a library whose correctness
guarantee depends on a test runner is a library you cannot check on a machine
that has no test runner:

```bash
python packages/vn-text-py/tests/test_vn_text.py   # stdlib only, nothing to install

pytest packages/vn-text-py/tests -q                     # if you have it
```

## API parity

`snake_case` is the Python convention, but `stripStroke` and `isVietnamese` are
exported as aliases so a call site can move between the Python and TypeScript
ports without renaming.

## Known port risk

`fold()` classifies characters using `str.isalnum()`, while the TypeScript port
uses `\p{L}\p{N}`. These agree across all 127 distinct characters in the current
corpus, measured by `scripts/verify-port-parity.mjs`. They are not guaranteed to
agree for every code point; the conformance suite is how a divergence gets
caught.

## Scope

Deliberately narrow. Tone-mark canonicalisation (`hóa` vs `hòa`) and fuzzy
matching for unaccented-keyboard input are not here. Sorting is not here
either -- it is [`vn-collate`](../vn-collate-py), which shares this
repository's conformance infrastructure. See the
[root README](../../README.md#what-this-is-not) for the reasoning.

## Licence

MIT
