Metadata-Version: 2.4
Name: addrnlp
Version: 1.6.91
Summary: Tokenize a Hong Kong address written in Traditional Chinese or English.
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: pandas>=2.0.0
Requires-Dist: openpyxl>=3.1.0

# AddrNlp

![Version](https://img.shields.io/badge/Version-1.6.91-blue?style=flat-square)
[![CIAO](https://img.shields.io/badge/Philosophy-CIAO%20(Caution%20%E2%80%A2%20Intentional%20%E2%80%A2%20Anti--fragile%20%E2%80%A2%20Over--engineered)-purple.svg)](https://github.com/cloudgen/ciao)
[![Python](https://img.shields.io/badge/Python-3.10%2B-blue?style=flat-square)]()

AddrNlp (`addr-nlp`) tokenizes a Hong Kong address written in Traditional Chinese or English. It reads the local token database and prints a JSON array. An ordinary object is `{token, type}`. An English lot marker or number range next to a street, and a Chinese street next to a lot marker, each become one object of type `street_with_lot_marker`. The longest dictionary token wins at each character, then the next shorter token. After every other step, an unmatched span that is an exact English or Chinese building name from the street catalog becomes `building`, and both names are added to the token database. A digit range such as `2-4` is `number_range`. A floor phrase such as `3rd Floor` is `floor`. A block such as `3B座` is `Block Number`. A house mark such as `HSE 1` is `House Number`. A tower mark such as `TOWER 1` is `Tower Number`. A phase mark such as `PHASE I` or `一期` is `Phase Number`. A Traditional Chinese street ending in `街` that the dictionary missed is still `street`.

**You** run one command from this checkout.  
**The command** reads `data/db/address_token.sqlite` and exits.  
**Not this:** a website, a listen port, a console script, or a tool that invents house numbers missing from the database.

| Includes | Excludes |
|----------|----------|
| Longest-match tokenization for Traditional Chinese and English | Browser client and HTTP API |
| Local SQLite dictionary of atomic address tokens | Overwriting raw address files |
| A visible `unmatched` gap when the database has no atom | Guessing a house number that was never stored |

| Step | What it means | What you type |
|------|----------------|---------------|
| Install the package | Package name `addrnlp`. No console script. The archive has no token database. | `python3 -m pip install addrnlp` |
| Install packages on this checkout | One-time for the builder. Needs CPython 3.10 or newer. | `python3 -m pip install -r requirements.txt` |
| Build the dictionary | Writes `data/db/address_token.sqlite` from the clean dictionaries. Skip when that file is already present. | `python -m etl.build_address_tokens` |
| Tokenize | Prints a JSON array and exits. | `python -m addrnlp "香港中環皇后大道中"` |

## Features

- One address becomes an ordered list. An ordinary object is `{token, type}`. Every combined lot and street object has `type` `street_with_lot_marker`. An English lot marker next to a street also has `lot_marker` and `street`. An English number range next to a street also has `street_lot_marker` and `street`. A Chinese street next to a lot marker also has `street` and `lot_marker`
- Types are the closed codes in the token database (`street`, `place`, `region`, `building`, and the rest of the 47-type catalog), plus `number_range` for a digit range and `floor` for a floor phrase
- Traditional Chinese matches the raw characters
- English matches a length-preserving fold (lower case, apostrophe as a space) and still prints your original spelling
- The fullwidth dash `－` is a separator, not a token
- A piece the database does not contain is returned as `unmatched`, except a digit range, a floor phrase, a digit `座` block, a Han street ending in `街`, and an English phrase that stops at `street` or `road`. Adjacent unmatched words separated only by whitespace become one token joined by a space. A comma keeps them apart. A facility ending on that token, such as a police station, a cooked-food market, or a market, uses that facility type. A church, police station, fire station, market, sports centre, street, car detection centre, car test centre, school, university, power station, pumping station, hospital, or clinic whose full name is not stored still takes that type in Chinese or in English of any letter case. A university stays `building`. A stored clinic building stays `building`

## Quick Installation

The package name is `addrnlp`. There is no console script and no self-update channel. You still run `python -m addrnlp`. The published archive does not include the token database or the raw registry files.

```bash
python3 -m pip install addrnlp
```

From this checkout, install the declared packages and run the module in place. An editable install keeps the local `data/` directory beside `etl/`:

```bash
python3 -m pip install -r requirements.txt
python3 -m pip install -e .
```

Needs CPython 3.10 or newer. Declared packages are pandas 2 and openpyxl 3.1 or newer. The tokenizer itself uses the Python standard library. `etl/config.py` resolves `data/db/address_token.sqlite` from the parent of the `etl` package. On this checkout that parent is the project root. A missing database exits non-zero and names `python -m etl.build_address_tokens`.

## Usage

From the project root:

```bash
python -m etl.build_address_tokens
python -m addrnlp "香港中環皇后大道中99號"
python -m addrnlp "Queen's Road Central"
python -m addrnlp --help
```

## Help

This checkout has no numbered menu and no graphical screen. The lines below are the opening of the live `python -m addrnlp --help` text at this version.

```text
AddrNlp 1.6.91 — tokenize a Hong Kong address (Traditional Chinese or English)

Usage:
  python -m addrnlp ADDRESS [ADDRESS ...]
  python -m addrnlp < addresses.txt
```

One address prints a pretty JSON array. Several addresses, or a pipe of one address per line, print one JSON array per line.

```bash
python -m addrnlp "九龍旺角亞皆老街" "沙田區沙頭角公路－石涌凹段"
```

If `data/db/address_token.sqlite` is missing, the command exits non-zero and tells you to run `python -m etl.build_address_tokens`.

## Examples

Traditional Chinese:

```bash
python -m addrnlp "香港中環皇后大道中99號"
```

```json
[
  {"token": "香港", "type": "region"},
  {"token": "中環", "type": "place"},
  {"token": "皇后大道中99號", "type": "street_with_lot_marker", "street": "皇后大道中", "lot_marker": "99號"}
]
```

`皇后大道中` is a street and `99號` is a lot marker. They sit next to each other, so they print as one object. A house number that was never stored, and that the lot pattern does not cover, stays `unmatched`. The command does not invent it.

English `2 Kwun Tsing Rd` prints `{ "token": "2, KWUN TSING RD", "type": "street_with_lot_marker", "lot_marker": "2", "street": "KWUN TSING RD" }`. English `2-4 Sheung Hei Street` prints `{ "token": "2-4, Sheung Hei Street", "type": "street_with_lot_marker", "street_lot_marker": "2-4", "street": "Sheung Hei Street" }`. A range with no following street stays `number_range`.

English:

```bash
python -m addrnlp "Queen's Road Central"
```

The stored atom is `queen s road central` with type `street`. The printed token keeps the original spelling.

## Platform Compatibility

| Surface | Status |
|---------|--------|
| POSIX Linux + CPython 3.10+ | Supported (developed on CPython 3.12) |
| Windows / macOS | Not claimed |
| Browser / HTTP / nginx | Removed |

## Related Projects

None published as a companion product.

## Contributing

Keep changes small. Do not strip CIAO Protection Zones on the longest-match scan. Dictionary atoms stay exact rows. `number_range`, `floor`, a digit `座` block, and a missed Han `街` street are the parse patterns in the domain file. Product behavior traces to `docs/requirements/requirement-domain-addrnlp.md`, `docs/requirements/requirement-system-architecture.md`, `docs/requirements/requirement-python-system-architecture.md`, and `docs/requirements/requirement-actor-role-subject.md`.

## License

No `LICENSE.md` is present. Reuse terms are unset. Do not treat this tree as MIT.

## Last Update

2026-10-31 — version **1.6.91** (the who-list is Actor, Role, and Subject. Help has no subject. The package name is `addrnlp`. The archive is the checkout modules. There is no console script. The token database stays out of the archive).
2026-10-21 — version **1.6.90** (`python -m etl.catalog_tokens` adds missing street and building names from the street catalog. `BONHAM TOWERS` and `般含閣` are stored as `building`. The token table is 68,056 rows).
2026-10-21 — version **1.6.89** (an unmatched span that is an exact street-catalog building name becomes `building`. `BONHAM TOWERS` and `般含閣` are the samples. Both spellings are inserted into the token database when that row is new).
2026-10-21 — version **1.6.88** (English street closers add Path, Terrace, Square, Circuit, Boulevard, Crescent, Close, Link, and Central. Chinese street closers add `台` and `臺`. `Zqxvbl Crescent` and `嶺青試作台` are `street`. `Central` alone stays unmatched).
2026-10-18 — version **1.6.87** (`棠蔭街山邊休憩處` on `深水埗石硤尾棠蔭街,棠蔭街山邊休憩處` is one `place`. The first `棠蔭街` stays `street`. `山邊休憩處` alone stays `park`).
2026-10-18 — version **1.6.86** (`Hill Road Flyover Sitting-out Area` and `Sitting-out Area under Flyover in Hill Road of Central and Western District` are each one `place`. `Hill Road` alone stays `street`. `Sitting-out Area` alone stays `unmatched`).
2026-10-18 — version **1.6.85** (`摩星嶺燒烤場` on `香港島薄扶林摩星嶺徑123號摩星嶺燒烤場` is one `place`. `摩星嶺徑123號` stays `street_with_lot_marker`. `摩星嶺` alone stays `place`).
2026-10-18 — version **1.6.84** (`聯興街休憩處` on `新界粉嶺聯和墟和睦路同聯興街交界聯興街休憩處` is one `park`. `同` is `joiner`. `交界` is `junction`. The first `聯興街` stays `street`).
2026-10-18 — version **1.6.83** (`大埔西沙路井頭村（近村公所）井頭村休憩處` keeps the first `井頭村` as `village` and `井頭村休憩處` as one `park`).
2026-10-18 — version **1.6.82** (`山道天橋休憩處` is one `place`. `山道` alone stays `street`. `天橋休憩處` stays `park`).
2026-10-18 — version **1.6.81** (`西貢半島` is one `place`. `西貢` alone stays `place`. `西貢獅子會自然教育中心` stays one `building`).
2026-10-18 — version **1.6.80** (`九龍窩打老道／渡船街交界` keeps `九龍` as `region`, both streets, `／` as `joiner`, and `交界` as `junction`).
2026-10-18 — version **1.6.79** (`西貢獅子會自然教育中心` on `香港新界西貢半島對面海蕉坑 西貢獅子會自然教育中心` is one `building`. The unfinished name without `心` stays split).
2026-10-18 — version **1.6.78** (`恆明街休憩處` is one `park`. `恆明街` alone stays `street`).
2026-10-18 — version **1.6.77** (`上水坑頭村第二號休憩處長者健身園地` is place `上水`, village `坑頭村`, park `第二號休憩處`, and park `長者健身園地`).
2026-10-18 — version **1.6.76** (`將軍澳南公園` on `西貢安禧街19號 將軍澳南公園（寵物公園）` is one `park`. `安禧街19號` stays `street_with_lot_marker`. `（寵物公園）` stays a joiner, a `park`, and a joiner).
2026-10-18 — version **1.6.75** (`寶琳休憩處寵物公園` on `西貢將軍澳寶琳北路寶琳休憩處寵物公園` is one `park`. `寶琳北路` stays `street`. `西貢` and `將軍澳` stay `place`).
2026-10-18 — version **1.6.74** (a building or unmatched name that ends as a municipal or government administration building is `administration_building`, in Chinese or in English of any letter case. The stored name list stops at 133 building rows).
2026-10-18 — version **1.6.73** (`環保大道寵物公園` on `西貢將軍澳環保大道,環保大道寵物公園` is one `park`. `西貢` and `將軍澳` stay `place`. The first `環保大道` stays `street`).
2026-10-18 — version **1.6.72** (a building or unmatched name that ends as a bus terminus, a bus station, a bus stop, a minibus terminus, or a public transport interchange is `bus_stop`, in Chinese or in English of any letter case. The printed list is 500 of 534 attested bilingual names).
2026-10-18 — version **1.6.71** (`市鎮公園北兒童遊樂場` is one `park`. A donor and the facility suffix `公園` stay inside that playground name. A stored building before a playground stays its own token).
2026-10-18 — version **1.6.70** (`Harbour Plaza Resort City` on `18 Tin Yan Road` stays one `hotel` and keeps its building id. `Tin Shui Wai` stays `place`).
2026-10-18 — version **1.6.69** (a building or unmatched name that ends as an industrial building or a factory building is `industrial_building`, in Chinese or in English of any letter case. The printed list is 500 attested bilingual names).
2026-10-18 — version **1.6.68** (`嘉湖海逸酒店` on `天恩路18號` stays one `hotel` and keeps its building id. A donor inside the lodging stem does not split the name. `天水圍` stays `street`).
2026-10-18 — version **1.6.67** (`悅品天秀酒店 ‧ 元朗` on `天秀路8號` is one `hotel` and keeps its building id. Spaces may sit around the place separator).
2026-10-18 — version **1.6.66** (a building or unmatched name that ends as a home for the aged is `aged_home`, in Chinese or in English of any letter case. The stored name list stops at 110 keyword rows).
2026-10-18 — version **1.6.65** (a building or unmatched hotel, hostel, or inn is `hotel`. A name that ends as a resort is `resort`, in Chinese or in English of any letter case).
2026-10-18 — version **1.6.64** (a building or unmatched name that ends as a godown, warehouse, cold store, or explosives depot is `goods_depot`, in Chinese or in English of any letter case).
2026-10-18 — version **1.6.63** (a building or unmatched name that ends as a petrol, gas, or filling station is `gas_station`. The token table has no stored gas-station name).
2026-10-18 — version **1.6.62** (a building or unmatched name that ends as a commercial building or commercial centre is `commercial_building`, in Chinese or in English of any letter case).
2026-10-18 — version **1.6.61** (a building or unmatched name that ends as a park or a playground is `park`. A garden ending stays `community`).
2026-10-18 — version **1.6.60** (an unstored nursery or children's home, in Chinese or in English of any letter case, is `children_home`. The stored name list stops at 29 tails).
2026-10-18 — version **1.6.59** (a forbidden slang word mixed into unknown Chinese or randomly cased English stays `bad_slang`, and the command exits 1 with no JSON array).
2026-10-18 — version **1.6.58** (an unstored sports-centre name, in Chinese or in English of any letter case, is `sports_centre`).
2026-10-18 — version **1.6.57** (a sewage or drainage pumping station whose full name is not stored is still `pump_house`, in Chinese or in English of any letter case).
2026-10-18 — version **1.6.56** (a forbidden slang word stored as `bad_slang`, such as `屌你老母`, makes the command exit 1 with no JSON array).
2026-10-18 — version **1.6.55** (a church, police station, fire station, market, sports centre, street, car detection centre, car test centre, school, university, power station, hospital, or clinic whose full name is not in the token database still takes that type, in Chinese or in English of any letter case).
2026-10-18 — version **1.6.54** (adjacent unmatched words separated only by whitespace become one token joined by a space, then the facility tails read that token).
2026-10-18 — version **1.6.53** (a school name that is not in the token database, such as `青松嶺實驗小學` or `Foo Random Primary School`, is one `school` token).
2026-10-18 — version **1.6.52** (English parse regexes match upper case, lower case, and mixed case. `Woosong street`, `g/f`, `67c`, and `blk h` stay one token).
2026-10-18 — version **1.6.51** (a lowercase phrase stops at the first `street` or `road` suffix, so `woosong street` is `street`).
2026-10-18 — version **1.6.50** (a `place` or `street` immediately before a facility joins that facility).
2026-10-18 — version **1.6.49** (adjacent unmatched words that form a testing centre become one `testing_centre` token).
2026-10-18 — version **1.6.47** (a `building` or `unmatched` token that ends as a fire station or fireboat station is `fire_station`).
2026-10-18 — version **1.6.46** (a `building` or `unmatched` token that ends as a theatre, a stadium or sports ground, or a city hall, town hall, or civic centre is `theatre`, `stadium`, or `city_hall`).
2026-10-18 — version **1.6.45** (a `building` or `unmatched` token that ends as a police station or police headquarters is `police_station`).
2026-10-18 — version **1.6.44** (a `building` or `unmatched` token that ends as a power station, substation, or voltage station is `power_station`).
2026-10-18 — version **1.6.43** (a `building` or `unmatched` token that ends as a vehicle detention or examination centre is `detention_centre`).
2026-10-18 — version **1.6.42** (a `building` or `unmatched` token that ends as a temporary market is `temp_market`).
2026-10-18 — version **1.6.41** (a `building` or `unmatched` token that ends as a cooked-food market is `food_mall`).
2026-10-18 — version **1.6.38** (a `building` or `unmatched` token that ends as a water, outdoor, or sea activity centre is `activity_centre`).
2026-10-18 — version **1.6.37** (`police_station` and `fire_station` are facility types. `元朗警署` is `police_station`. `上水消防局` is `fire_station`).
2026-10-18 — version **1.6.36** (a `building` or `unmatched` token that ends in `游泳池` or `swimming pool` is `swimming_pool`).
2026-10-18 — version **1.6.35** (a `building` or `unmatched` token that ends in `公共貨物裝卸區`, `公眾貨物裝卸區`, or in `public cargo working area` or `public cargo handling area`, is `public_area`).
2026-10-18 — version **1.6.34** (a `building` or `unmatched` token that ends in `基地` or in an English operations or activity base phrase is `operations_base`).
2026-10-18 — version **1.6.33** (a `building` or `unmatched` token that ends in `街市`, `市場`, `market`, `marketplace`, or `bazaar` is `market`).
2026-10-18 — version **1.6.31** (a `building` or `unmatched` token that ends in `醫院` or `hospital` is `hospital`).
2026-10-18 — version **1.6.30** (a shop word before `2/F` stays unmatched, and `2/F` stays `floor`. `SHOP 2` stays `Shop Number`).
2026-10-18 — version **1.6.29** (a `building` or `unmatched` token that ends in `堂` or `Chapel` is `chapel`).
2026-10-18 — version **1.6.28** (`基督君王小堂` is a building atom and building `objectid` `51355`).
2026-10-18 — version **1.6.27** (`Market` closes a name as `market`, so `Tai Kiu Market` is one token. `College` closes a building).
2026-10-18 — version **1.6.26** (a remaining lot marker may be `Block Number`, `House Number`, `Tower Number`, `Phase Number`, `Villa Number`, or `Stage Number`).
2026-10-18 — version **1.6.25** (a building or facility token gains `object_id` from the highest-scoring `building` row; a tie keeps the smaller id).
2026-10-18 — version **1.6.24** (a building that ends with the stored atom `醫院`, or with the English word `hospital`, is emitted as `facility_suffix`).
2026-10-18 — version **1.6.23** (a lot marker that starts with Shop is `Shop Number`, and one that starts with Block is `Block Number`).
