Metadata-Version: 2.4
Name: ukvocab
Version: 0.3.0
Summary: The published Ukrainian vocabulary data from jakelawrence.xyz/research/ukrainian-frequency, offline: the top-12,000 lemma frequency list, the validated PULS CEFR overlay, the Borodin A1 and A2 lexical minimums, their crosswalk, and the B1 enrichment tables.
Author: Jake Lawrence
License-Expression: CC-BY-NC-SA-4.0
Project-URL: Homepage, https://www.jakelawrence.xyz/research/ukrainian-frequency
Project-URL: Source, https://github.com/jake0lawrence/jakelawrence.xyz/tree/main/packages/ukvocab
Project-URL: Data, https://www.jakelawrence.xyz/downloads/ukrainian-frequency/
Keywords: ukrainian,vocabulary,frequency list,CEFR,lexical minimum,language learning,corpus
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Education
Classifier: Intended Audience :: Science/Research
Classifier: Natural Language :: Ukrainian
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Topic :: Education
Classifier: Topic :: Text Processing :: Linguistic
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSES.md
Requires-Dist: pyarrow>=14
Provides-Extra: test
Requires-Dist: pytest>=7; extra == "test"
Dynamic: license-file

# ukvocab

The published Ukrainian vocabulary data from
[jakelawrence.xyz/research/ukrainian-frequency](https://www.jakelawrence.xyz/research/ukrainian-frequency),
installable and offline: the top-12,000 lemma frequency list over UberText 2.0,
the validated PULS CEFR overlay (A1 to C1), the Borodin A1 and A2 lexical
minimums, the crosswalk between the minimums and the profile, and the B1
enrichment tables (aspect relations, case government, collocations).

```python
import ukvocab
ukvocab.lookup("п'ять")["level"]                       # 'A1', with rank, source and enrichment beside it
ukvocab.coverage(["в", "школа", "йти"], "A2")["coverage"]  # share of tokens a validated-A2 learner knows
```

## What you get

| function | returns |
|---|---|
| `lookup(lemma, pos=None)` | one lemma: `level`, `level_source`, `rank`, the frequency figures, every other reading of the spelling, minimum membership, the crosswalk class, and the B1 `enrichment` (aspect partners, government, collocations); `None` when no table knows the string |
| `levels(lemmas)` | batch: input string to `{level, level_source, rank, pos, source}` |
| `coverage(lemmas, known_set)` | token coverage of a lemmatized text against a known set: a level (`"A2"` means every validated lemma at or below A2), `"borodin-a1"` / `"borodin-a2"`, an int N (top N of the list), or your own set of lemmas; reports the unknowns by frequency |
| `sources()` | version, snapshot date, license, attribution, citation and SHA-256 per dataset, the combined license and why, and what is deliberately not shipped |

Every return value carries the dataset, version and snapshot it came from.
Data is Parquet, loaded lazily on first use, never on import. The only
dependency is `pyarrow`.

Two things the package does not do, on purpose. It does not lemmatize: pass
lemmas in the UberText / VESUM convention (infinitive for verbs, nominative
singular for nouns); the site's `POST /api/lemmatize` is the hosted lemmatizer
matched to this list. And it ships no predicted CEFR levels: only the profile's
validated levels are here, and a lemma the profile does not cover gets
`level: None` rather than a guess.

## Provenance and honesty

The data tier is generated from the site's published download files by
`scripts/ukrainian-frequency/build-package.py` in the source repository and is
never edited by hand; the tests assert that every table's row count and hash
match the published file exactly. `sources()` is the manifest that build
wrote. The PULS overlay is a dated snapshot (2026-07-23) of a profile that is
still being extended, redistributed with its authors' permission. The join
onto the frequency list is part-of-speech aware; ten declared cross-tag rules
recover the words the two tag conventions merely label differently, and each
row's `join` field says how it matched, so the strict join is one filter away.

## License

The combined package is **CC-BY-NC-SA 4.0**, because the PULS levels are
NonCommercial and share-alike at the authors' decision (2026-09-24) and they run
through most tables. The frequency list and the two Borodin minimums are CC-BY
4.0 on their own, and the government and collocation tables CC-BY-SA 4.0; drop
the PULS level columns and NonCommercial does not apply. `LICENSES.md` has the
per-dataset terms and every attribution.

## Citation

```
Lawrence, J. (2026). ukvocab: the Ukrainian Frequency datasets (top-12,000 lemma
list, validated PULS CEFR overlay, Borodin A1/A2 minimums, crosswalk, B1
enrichment tables), version 0.2.0. jakelawrence.xyz/research/ukrainian-frequency.

Upstream: Chaplynskyi, D. (2023). Introducing UberText 2.0: A Corpus of Modern
Ukrainian at Scale. Proc. 2nd UNLP, ACL. | Synchak, O., Starko, V., Burak, M.,
Svystun, M. (2025). Corpus-Based Vocabulary Profiling for Ukrainian: From
Lexical Analysis to CEFR. eLex 2025; PULS, puls.peremova.org. | Borodin, K. and
Turkevych, O. 1000 i 1 slovo, A1. https://doi.org/10.18452/28236 | Borodin, K.
and Lazarenko, O. (2026). 1000 i 1 slovo, A2. https://doi.org/10.11584/opus4-1448
```

Archived, DOI-minted copies of each dataset are listed in `sources()` under
`conceptDoi` once they exist; cite the concept DOI for the dataset you used.
