Metadata-Version: 2.5
Name: localis
Version: 3.0.0
Summary: Offline reference data for countries, subdivisions, cities, macroregions, currencies, scripts and languages, merged from ISO standards, GeoNames, Unicode CLDR and Wikidata, with lookups, filtering and typo-tolerant search.
Project-URL: homepage, https://github.com/dstoffels/localis
Project-URL: repository, https://github.com/dstoffels/localis
Project-URL: changelog, https://github.com/dstoffels/localis/blob/main/CHANGELOG.md
Project-URL: data changelog, https://github.com/dstoffels/localis/blob/main/DATA_CHANGELOG.md
Project-URL: funding, https://github.com/sponsors/dstoffels
Author-email: danimal <dan.stoffels@gmail.com>
License-Expression: MIT AND LGPL-2.1-or-later AND CC-BY-4.0 AND Unicode-3.0 AND CC0-1.0
License-File: LICENSE
License-File: LICENSES/CC-BY-4.0.txt
License-File: LICENSES/CC0-1.0.txt
License-File: LICENSES/LGPL-2.1-or-later.txt
License-File: LICENSES/Unicode-3.0.txt
Keywords: 3166,3166-1,3166-2,3166-3,city,cldr,country,country codes,currency,fuzzy-search,gazetteer,geocoding,geodata,geography,geonames,iso,iso-15924,iso-3166,iso-4217,iso-639,iso-639-3,language,m49,macroregion,offline,province,region,script,state,subdivision,wikidata
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: GIS
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Typing :: Typed
Requires-Python: >=3.11
Requires-Dist: rapidfuzz>=3.13.0
Description-Content-Type: text/markdown

# localis

Offline reference data for countries, subdivisions, cities, macroregions, currencies, scripts and languages, merged from ISO standards, GeoNames, Unicode CLDR and Wikidata. Records can be looked up by code or ID, filtered by field or searched by name with typo tolerance.

## Contents

- **<stat key="data.countries.total:int">281</stat> countries**, <stat key="data.countries.historic:int">31</stat> historic, with their currencies, official languages, macroregions and continent
- **<stat key="data.subdivisions.total:int">51,827</stat> subdivisions** from ISO 3166-2 and GeoNames, each with its parent and country
- **<stat key="data.cities.total:int">236,866</stat> cities** from GeoNames, each with its full subdivision chain, population and coordinates
- **<stat key="data.macroregions.total:int">34</stat> macroregions** from Unicode CLDR: <stat key="data.macroregions.regions:int">5</stat> regions, <stat key="data.macroregions.subregions:int">23</stat> subregions and <stat key="data.macroregions.groupings:int">6</stat> groupings
- **<stat key="data.currencies.total:int">178</stat> currencies** and funds from ISO 4217
- **<stat key="data.scripts.total:int">226</stat> scripts** from ISO 15924
- **<stat key="data.languages.total:int">7,923</stat> languages** from ISO 639-3, with their scripts

Search and name filters also match aliases: colloquial, historic and alternate names. With a typo in the query, the intended record ranks first for <stat key="bench.registries.countries.accuracy.top1_pct:pct">99.3%</stat> of countries, <stat key="bench.registries.subdivisions.accuracy.top1_pct:pct">93.4%</stat> of subdivisions and <stat key="bench.registries.cities.accuracy.top1_pct:pct">93.4%</stat> of cities.


## Installation

```bash
pip install localis

uv add localis
```

### Quick Start

```python
import localis

# Exact lookups by code
country = localis.countries.lookup("US")
print(country.name)  # "United States"

state = localis.subdivisions.lookup("US-CA")
print(state.name)  # "California"

# Filters: exact matches, combined with AND
cities = localis.cities.filter(country="US", subdivision="California", limit=10)

# Typo-tolerant search: (result, score) pairs, best first
results = localis.countries.search("Austrlia")
print(results[0][0].name)  # "Australia"
```

## Versioning

localis follows [Semantic Versioning](https://github.com/dstoffels/localis/blob/main/docs/versioning.md). Every monthly data refresh is a patch release: records are added, renamed and removed with their sources, counted per dataset in the [CHANGELOG](https://github.com/dstoffels/localis/blob/main/CHANGELOG.md) and listed in the [data changelog](https://github.com/dstoffels/localis/blob/main/DATA_CHANGELOG.md). IDs are renumbered by every refresh, so store [key](#key), not `id`. Upgrading from 2.1? The CHANGELOG lists [what changed in 3.0](https://github.com/dstoffels/localis/blob/main/CHANGELOG.md#upgrading-from-210).

## Querying

Each dataset is a registry sharing one query API.

| Registry | `get` | `lookup` | `filter` | `search` | Iteration |
|---|---|---|---|---|---|
| `countries` | ✓ | ✓ | ✓ | ✓ | ✓ |
| `subdivisions` | ✓ | ✓ | ✓ | ✓ | ✓ |
| `cities` | ✓ | ✓ | ✓ | ✓ | ✓ |
| `macroregions` | ✓ | ✓ | | | ✓ |
| `currencies` | ✓ | ✓ | ✓ | ✓ | ✓ |
| `scripts` | ✓ | ✓ | ✓ | ✓ | ✓ |
| `languages` | ✓ | ✓ | ✓ | ✓ | ✓ |

Lookups, filters and search ignore case and accents, so "sao paulo" finds São Paulo and "strasse" finds Straße.

### get

**Returns:** the entity with that localis ID, or `None`. IDs are only valid within the installed version; to store a reference, use [key](#key).

```python
localis.countries.get(1)
```

### lookup

Resolves a record by an identifier other than its localis ID. Identifiers only: "UK" isn't an ISO code, so find it with `filter(name=...)` or `search()`.

**Returns:** the entity, or `None`. A key that isn't a string or an int raises `TypeError`.

| Registry | Identifiers |
|---|---|
| `countries` | alpha-2 (`"GB"`), alpha-3 (`"GBR"`), numeric (`826`), a historic entry only by its `alpha_4` (`"CSHH"`, see [Historic Countries](#historic-countries)) |
| `subdivisions` | ISO 3166-2 code (`"US-CA"`), GeoNames code (`"US.CA"`) |
| `cities` | GeoNames ID (`5128581`) |
| `macroregions` | code (`"155"`, `"EU"`), name (`"Western Europe"`); M49 codes are zero-padded strings, so `lookup(9)` finds nothing |
| `currencies` | alpha-3 (`"EUR"`), numeric (`978`) |
| `scripts` | alpha-4 (`"Cyrl"`), numeric (`220`) |
| `languages` | ISO 639-3 (`"deu"`), ISO 639-1 (`"de"`), ISO 639-2/B (`"ger"`) |

```python
# "GB", "GBR" and 826 all find the United Kingdom
localis.countries.lookup("GBR")

# a numeric code or GeoNames ID as an int
localis.cities.lookup(5128581)
```

### filter

Exact matches on any value a field indexes. Using multiple fields combines the conditions with a logical AND. Pass `localis.MISSING` to match records with no value in a field (`None` ignores the field). A field the registry doesn't have raises `TypeError`, as does a call with no field.

**Returns:** a list of entities sorted by name. `limit` defaults to every match and must be at least 1.

#### Fields by Registry

**Countries**
- `name` (name, official name, common name or alias)
- `macroregion` (a region, subregion or grouping, by name or code)
- `currency` (by name or alpha-3)
- `language` (an official language, by name or ISO 639 code)
- `continent` (by name or GeoNames code)

**Subdivisions**
- `name` (name or alias)
- `type`
- `country` (name, common name, alpha-2, alpha-3 or numeric, as `76` or `"076"`)
- `admin_level` (0 = non-administrative groupings, 1 = states/provinces, 2 = counties/districts, 3 = divisions below those)

**Cities**
- `name`
- `country` (name, common name, alpha-2 or alpha-3)
- `subdivision` (any subdivision in the city's chain, by name, ISO code or its suffix (`"CA"`), or GeoNames code)

**Currencies**
- `name`

**Scripts**
- `name` (name or alias)

**Languages**
- `name` (name, inverted name or alias)
- `scope`
- `type`
- `script` (by alpha-4, name or alias, primary or secondary)

```python
# aliases and abbreviations match too
localis.countries.filter(name="UK")

# countries whose legal tender includes the euro
localis.countries.filter(currency="EUR")

# fields combine with AND: Germany's states
localis.subdivisions.filter(country="DE", admin_level=1)

# GeoNames-only subdivisions, which have no ISO type
localis.subdivisions.filter(type=localis.MISSING)
```

### search

Typo-tolerant matching on names and aliases. A subdivision or city query can add context after the name: a subdivision's parent or country, or a city's first-level subdivision or country.

**Returns:** a list of `(entity, score)` pairs, best first, scored 0 to 1; cities with the same score rank larger first. `limit` defaults to 10 and must be at least 1. For cities, `population_sort=True` reorders the best `limit` matches by population, largest first.

```python
# Germany first, then weaker matches such as Guernsey
localis.countries.search("Germny", limit=5)

# context after the name narrows the match
localis.cities.search("Springfeld, Illinois")

# the best matches, largest city first
localis.cities.search("Springfield", population_sort=True)
```

### Iteration and len()

```python
for country in localis.countries:
    print(country.name)

total = len(localis.subdivisions)
```

> ℹ️ Historic countries are left out unless included (see [Historic Countries](#historic-countries)), and a [population threshold](#population-threshold) narrows every cities query.

## Entities

Results are typed dataclasses with `to_dict()` and `json()`; `str()` gives the JSON. Each is built fresh, so it's yours to change, and hashable by its type and `id`. A record nested in another, such as `subdivision.country`, is its base form (`CountryBase`, `SubdivisionBase`, ...), keeping the fields marked Base below; one that carries facts about the relationship adds them (`CountryLanguage`, `LanguageScript`). Every entity type, `Entity`, `Missing` and the registry classes import from `localis` for type annotations.

```python
# every field, nested records included
localis.countries.lookup("US").to_dict()

# "USA", from the nested CountryBase
localis.subdivisions.lookup("US-CA").country.alpha3
```

### key

IDs change between versions (`localis.__version__` gives the installed one). Store `key` instead: `lookup()` resolves it in any later version. Nested records carry it too.

```python
# 5128581, safe to persist
saved = city.key

# the same city, in this version or a later one
city = localis.cities.lookup(saved)
```

| Registry | `key` |
|---|---|
| `countries` | `alpha2`, or `historic.alpha_4` for a historic entry |
| `subdivisions` | `iso_code`, or `geonames_code` for a subdivision ISO doesn't list |
| `cities` | `geonames_id` |
| `macroregions` | `code` |
| `currencies` | `alpha3` |
| `scripts` | `alpha4` |
| `languages` | `alpha3` (ISO 639-3) |

### Countries

#### Historic Countries

ISO 3166-3 withdrawn countries are left out of `filter()`, `search()`, iteration and `len()` unless included with `set_include_historic(True)` (`include_historic` reads the setting). `get()` always finds them, and `lookup()` finds them by `alpha_4` only. `len(localis.countries)` is <stat key="data.countries.current:int">250</stat> by default and <stat key="data.countries.total:int">281</stat> with them.

```python
# Czechoslovakia
localis.countries.lookup("CSHH")

# None: a reused code never resolves a historic entry
localis.countries.lookup("CS")
```

#### Country Object

A ✓ under Base marks a field the nested `CountryBase` also has.

| Field | Type | Example (`"US"`) | Notes | Base |
|---|---|---|---|---|
| `id` | `int` | | localis ID, valid within this version | ✓ |
| `key` | `str` | `"US"` | stable reference to store; `alpha_4` for a historic entry | ✓ |
| `name` | `str` | `"United States"` | ISO 3166-1 name, as published | ✓ |
| `official_name` | `str \| None` | `"United States of America"` | ISO 3166-1 official name; `None` where ISO has none | |
| `common_name` | `str \| None` | `None` | Debian iso-codes' everyday name where it differs, such as "South Korea" for "Korea, Republic of" | |
| `alpha2` | `str` | `"US"` | | ✓ |
| `alpha3` | `str \| None` | `"USA"` | | ✓ |
| `numeric` | `int \| None` | `840` | ISO 3166-1 numeric code; `None` for Kosovo, which has no ISO assignment | |
| `geonames_id` | `int \| None` | `6252001` | | ✓ |
| `aliases` | `list[str]` | | alternate names from GeoNames and Wikidata | |
| `flag` | `str \| None` | `"🇺🇸"` | Unicode flag emoji | |
| `historic` | `HistoricInfo \| None` | `None` | set only for withdrawn ISO 3166-3 countries | |
| `continent` | `Continent \| None` | North America | GeoNames' continent (see [Continent](#continent)); `None` for a country GeoNames doesn't list | |
| `macroregions` | `list[MacroregionBase]` | [Americas, Northern America] | CLDR path, region then subregion; `[]` for most historic countries | |
| `groupings` | `list[MacroregionBase]` | [North America, United Nations] | CLDR groupings the country belongs to | |
| `currencies` | `list[CurrencyBase]` | [US Dollar] | legal tender in use, per CLDR, in CLDR's order; `[]` for historic countries | |
| `languages` | `list[CountryLanguage]` | [English, Spanish, Hawaiian] | languages with an official status, per CLDR, by population share; `[]` for historic countries (see [CountryLanguage](#countrylanguage)) | |

##### HistoricInfo

| Field | Type | Example (`"CSHH"`) | Notes |
|---|---|---|---|
| `alpha_4` | `str` | `"CSHH"` | ISO 3166-3 withdrawal code, unique to the entry |
| `withdrawal_date` | `str` | `"1993-01-01"` | |
| `comment` | `str \| None` | | ISO's comment |

##### Continent

GeoNames enrichment, not an authoritative standard, and separate from macroregions, whose memberships can differ (GeoNames puts Cyprus in Europe, CLDR in Western Asia).

| Field | Type | Example (`"US"`) | Notes |
|---|---|---|---|
| `name` | `str` | `"North America"` | |
| `geonames_code` | `str` | `"NA"` | GeoNames' continent code, not an ISO code |
| `geonames_id` | `int` | `6255149` | |

### Subdivisions

#### Subdivision Object

A ✓ under Base marks a field the nested `SubdivisionBase` also has.

| Field | Type | Example (`"US-CA"`) | Notes | Base |
|---|---|---|---|---|
| `id` | `int` | | localis ID, valid within this version | ✓ |
| `key` | `str` | `"US-CA"` | stable reference to store; the GeoNames code where ISO doesn't list the subdivision | ✓ |
| `name` | `str` | `"California"` | ISO 3166-2 name where ISO lists the subdivision, otherwise GeoNames' | ✓ |
| `iso_code` | `str \| None` | `"US-CA"` | `None` for a GeoNames-only subdivision | ✓ |
| `geonames_code` | `str \| None` | `"US.CA"` | GeoNames admin code; `None` for an ISO-only subdivision | ✓ |
| `geonames_id` | `int \| None` | `5332921` | `None` for an ISO-only subdivision | ✓ |
| `type` | `str \| None` | `"State"` | ISO type; `None` for a GeoNames-only subdivision | ✓ |
| `admin_level` | `int` | `1` | 0 = non-administrative grouping, 1 = top-level, 2 = second-level, 3 = below that | ✓ |
| `parent` | `SubdivisionBase \| None` | `None` | the subdivision it sits in | |
| `country` | `CountryBase` | United States | | |
| `aliases` | `list[str]` | | alternate names | |

### Cities

#### Population Threshold

Narrows the cache and every index to cities of at least that population (`population_threshold` reads it). Set it before first access; changing it later rebuilds the cache and indexes.

```python
localis.cities.set_population_threshold(15000)

# Builds the cache with only cities of 15,000 or more on next access
len(localis.cities)

# back to every city
localis.cities.set_population_threshold(None)
```

#### City Object

| Field | Type | Example (`5128581`) | Notes |
|---|---|---|---|
| `id` | `int` | | localis ID, valid within this version |
| `key` | `int` | `5128581` | stable reference to store, the GeoNames ID |
| `geonames_id` | `int` | `5128581` | |
| `name` | `str` | `"New York"` | GeoNames' name, or its ASCII form where the name is in another script |
| `subdivisions` | `list[SubdivisionBase]` | | the city's full subdivision chain, ordered by `admin_level` ascending |
| `country` | `CountryBase` | United States | |
| `population` | `int` | | GeoNames population; `0` where unknown |
| `lat` | `float` | `40.71427` | |
| `lng` | `float` | `-74.00597` | |

### Macroregions

#### Macroregion Object

A ✓ under Base marks a field the nested `MacroregionBase` also has.

| Field | Type | Example (`"155"`) | Notes | Base |
|---|---|---|---|---|
| `id` | `int` | | localis ID, valid within this version | ✓ |
| `key` | `str` | `"155"` | stable reference to store, the code | ✓ |
| `name` | `str` | `"Western Europe"` | CLDR English name | ✓ |
| `code` | `str` | `"155"` | M49 numeric code as a zero-padded string, or CLDR's letter code (`"QO"`, `"EU"`) | ✓ |
| `type` | `MacroregionType` | `"subregion"` | `"region"`, `"subregion"` or `"grouping"` | ✓ |
| `parent` | `MacroregionBase \| None` | Europe | a subregion's region, or the region CLDR files a grouping under | |

### Currencies

Every ISO 4217 code, funds, precious metals and testing codes included. Countries' legal tender comes from Unicode CLDR (see [Country Object](#country-object)).

#### Currency Object

A ✓ under Base marks a field the nested `CurrencyBase` also has.

| Field | Type | Example (`"EUR"`) | Notes | Base |
|---|---|---|---|---|
| `id` | `int` | | localis ID, valid within this version | ✓ |
| `key` | `str` | `"EUR"` | stable reference to store, the alpha-3 | ✓ |
| `name` | `str` | `"Euro"` | ISO 4217 name, as published | ✓ |
| `alpha3` | `str` | `"EUR"` | | ✓ |
| `numeric` | `int \| None` | `978` | ISO 4217 numeric code | |

### Scripts

Every ISO 15924 code, special and private-use codes included (`"Zyyy"`, `"Qaaa"`). Unicode CLDR's English names are aliases, so "Han" finds "Han (Hanzi, Kanji, Hanja)".

#### Script Object

A ✓ under Base marks a field the nested `ScriptBase` also has.

| Field | Type | Example (`"Deva"`) | Notes | Base |
|---|---|---|---|---|
| `id` | `int` | | localis ID, valid within this version | ✓ |
| `key` | `str` | `"Deva"` | stable reference to store, the alpha-4 | ✓ |
| `name` | `str` | `"Devanagari (Nagari)"` | ISO 15924 name, as published | ✓ |
| `alpha4` | `str` | `"Deva"` | | ✓ |
| `numeric` | `int \| None` | `315` | ISO 15924 numeric code | |
| `aliases` | `list[str]` | ["Devanagari"] | Unicode CLDR's English names for the code, where they differ from ISO's | |

### Languages

Every ISO 639-3 code, macrolanguages and special codes (`"und"`, `"mul"`) included. Scripts come from Unicode CLDR, which covers <stat key="data.languages.with_scripts:int">820</stat> languages; the rest have none.

```python
# [LanguageScript(alpha4="Latn", secondary=False, ...)]
localis.languages.lookup("de").scripts

# Hong Kong's official languages, by population share
localis.countries.lookup("HK").languages
```

#### Language Object

A ✓ under Base marks a field the nested `LanguageBase` also has.

| Field | Type | Example (`"deu"`) | Notes | Base |
|---|---|---|---|---|
| `id` | `int` | | localis ID, valid within this version | ✓ |
| `key` | `str` | `"deu"` | stable reference to store, the ISO 639-3 code | ✓ |
| `name` | `str` | `"German"` | ISO 639-3 name, as published | ✓ |
| `alpha3` | `str` | `"deu"` | ISO 639-3 code | ✓ |
| `alpha2` | `str \| None` | `"de"` | ISO 639-1 code, for the <stat key="data.languages.with_alpha2:int">184</stat> languages that have one | ✓ |
| `bibliographic` | `str \| None` | `"ger"` | ISO 639-2/B code, where it differs from the 639-3 one | |
| `scope` | `LanguageScope` | `"individual"` | `"individual"`, `"macrolanguage"` or `"special"` | |
| `type` | `LanguageType` | `"living"` | `"living"`, `"extinct"`, `"historical"`, `"constructed"` or `"special"` | |
| `inverted_name` | `str \| None` | `None` | ISO's name with the qualifier moved last, such as "Arabic, Algerian Saharan" | |
| `aliases` | `list[str]` | ["Austrian German", ...] | Unicode CLDR's English names for the language and its regional and script forms, where they differ from ISO's | |
| `scripts` | `list[LanguageScript]` | [Latin] | per CLDR, primary scripts first; see [LanguageScript](#languagescript) | |

#### LanguageScript

A script as one language uses it: every `ScriptBase` field plus `secondary`.

| Field | Type | Notes |
|---|---|---|
| `secondary` | `bool` | CLDR's rule: `True` when the language isn't a modern language or the script isn't a modern script, such as Arabic written in Syriac or anything in Sanskrit |

`languages.filter(script=...)` matches primary and secondary scripts alike.

#### CountryLanguage

A language as one country recognizes it: every `LanguageBase` field plus three from Unicode CLDR, one per language and script with an official status.

| Field | Type | Example (Hong Kong's Chinese) | Notes |
|---|---|---|---|
| `status` | `LanguageStatus` | `"official"` | `"official"`, `"regional"` (official in part of the country) or `"de_facto"` (official in practice, such as English in the US) |
| `population_percent` | `float \| None` | `95.0` | CLDR's estimate of the population using it; shares overlap, since people use several languages |
| `script` | `ScriptBase \| None` | Han (Traditional variant) | the script CLDR gives the status for, so a language can appear once per script; `None` where CLDR names none |

## Performance

### Caching

Registries and their indexes load on first use; `force_cache()` loads one upfront. Each cell is load time / memory for that registry alone.

| Registry | Records | Dataset | Lookup index | Filter index | Search index | Combined |
|---|---|---|---|---|---|---|
| Macroregions | <stat key="data.macroregions.total:int">34</stat> | <stat key="footprint.registries.macroregions.dataset.time_ms:load">< 1ms</stat> / <stat key="footprint.registries.macroregions.dataset.memory_bytes:size">6KB</stat> | <stat key="footprint.registries.macroregions.lookup_index.time_ms:load">< 1ms</stat> / <stat key="footprint.registries.macroregions.lookup_index.memory_bytes:size">5KB</stat> | n/a | n/a | **<stat key="footprint.registries.macroregions.combined.time_ms:load">< 1ms</stat> / <stat key="footprint.registries.macroregions.combined.memory_bytes:size">11KB</stat>** |
| Currencies | <stat key="data.currencies.total:int">178</stat> | <stat key="footprint.registries.currencies.dataset.time_ms:load">< 1ms</stat> / <stat key="footprint.registries.currencies.dataset.memory_bytes:size">24KB</stat> | <stat key="footprint.registries.currencies.lookup_index.time_ms:load">< 1ms</stat> / <stat key="footprint.registries.currencies.lookup_index.memory_bytes:size">12KB</stat> | <stat key="footprint.registries.currencies.filter_index.time_ms:load">< 1ms</stat> / <stat key="footprint.registries.currencies.filter_index.memory_bytes:size">17KB</stat> | <stat key="footprint.registries.currencies.search_index.time_ms:load">< 1ms</stat> / <stat key="footprint.registries.currencies.search_index.memory_bytes:size">162KB</stat> | **<stat key="footprint.registries.currencies.combined.time_ms:load">~1ms</stat> / <stat key="footprint.registries.currencies.combined.memory_bytes:size">215KB</stat>** |
| Scripts | <stat key="data.scripts.total:int">226</stat> | <stat key="footprint.registries.scripts.dataset.time_ms:load">< 1ms</stat> / <stat key="footprint.registries.scripts.dataset.memory_bytes:size">40KB</stat> | <stat key="footprint.registries.scripts.lookup_index.time_ms:load">< 1ms</stat> / <stat key="footprint.registries.scripts.lookup_index.memory_bytes:size">16KB</stat> | <stat key="footprint.registries.scripts.filter_index.time_ms:load">< 1ms</stat> / <stat key="footprint.registries.scripts.filter_index.memory_bytes:size">24KB</stat> | <stat key="footprint.registries.scripts.search_index.time_ms:load">< 1ms</stat> / <stat key="footprint.registries.scripts.search_index.memory_bytes:size">203KB</stat> | **<stat key="footprint.registries.scripts.combined.time_ms:load">~2ms</stat> / <stat key="footprint.registries.scripts.combined.memory_bytes:size">282KB</stat>** |
| Languages | <stat key="data.languages.total:int">7,923</stat> | <stat key="footprint.registries.languages.dataset.time_ms:load">~9ms</stat> / <stat key="footprint.registries.languages.dataset.memory_bytes:size">1.6MB</stat> | <stat key="footprint.registries.languages.lookup_index.time_ms:load">~2ms</stat> / <stat key="footprint.registries.languages.lookup_index.memory_bytes:size">459KB</stat> | <stat key="footprint.registries.languages.filter_index.time_ms:load">~6ms</stat> / <stat key="footprint.registries.languages.filter_index.memory_bytes:size">1.1MB</stat> | <stat key="footprint.registries.languages.search_index.time_ms:load">~7ms</stat> / <stat key="footprint.registries.languages.search_index.memory_bytes:size">1.3MB</stat> | **<stat key="footprint.registries.languages.combined.time_ms:load">~24ms</stat> / <stat key="footprint.registries.languages.combined.memory_bytes:size">4.5MB</stat>** |
| Countries | <stat key="data.countries.total:int">281</stat> | <stat key="footprint.registries.countries.dataset.time_ms:load">~1ms</stat> / <stat key="footprint.registries.countries.dataset.memory_bytes:size">316KB</stat> | <stat key="footprint.registries.countries.lookup_index.time_ms:load">< 1ms</stat> / <stat key="footprint.registries.countries.lookup_index.memory_bytes:size">33KB</stat> | <stat key="footprint.registries.countries.filter_index.time_ms:load">~1ms</stat> / <stat key="footprint.registries.countries.filter_index.memory_bytes:size">198KB</stat> | <stat key="footprint.registries.countries.search_index.time_ms:load">~2ms</stat> / <stat key="footprint.registries.countries.search_index.memory_bytes:size">421KB</stat> | **<stat key="footprint.registries.countries.combined.time_ms:load">~5ms</stat> / <stat key="footprint.registries.countries.combined.memory_bytes:size">968KB</stat>** |
| Subdivisions | <stat key="data.subdivisions.total:int">51,827</stat> | <stat key="footprint.registries.subdivisions.dataset.time_ms:load">~58ms</stat> / <stat key="footprint.registries.subdivisions.dataset.memory_bytes:size">12.3MB</stat> | <stat key="footprint.registries.subdivisions.lookup_index.time_ms:load">~13ms</stat> / <stat key="footprint.registries.subdivisions.lookup_index.memory_bytes:size">3.6MB</stat> | <stat key="footprint.registries.subdivisions.filter_index.time_ms:load">~55ms</stat> / <stat key="footprint.registries.subdivisions.filter_index.memory_bytes:size">9.9MB</stat> | <stat key="footprint.registries.subdivisions.search_index.time_ms:load">~46ms</stat> / <stat key="footprint.registries.subdivisions.search_index.memory_bytes:size">11.9MB</stat> | **<stat key="footprint.registries.subdivisions.combined.time_ms:load">~173ms</stat> / <stat key="footprint.registries.subdivisions.combined.memory_bytes:size">37.7MB</stat>** |
| Cities | <stat key="data.cities.total:int">236,866</stat> | <stat key="footprint.registries.cities.dataset.time_ms:load">~285ms</stat> / <stat key="footprint.registries.cities.dataset.memory_bytes:size">25.8MB</stat> | <stat key="footprint.registries.cities.lookup_index.time_ms:load">~60ms</stat> / <stat key="footprint.registries.cities.lookup_index.memory_bytes:size">1.9MB</stat> | <stat key="footprint.registries.cities.filter_index.time_ms:load">~227ms</stat> / <stat key="footprint.registries.cities.filter_index.memory_bytes:size">43.7MB</stat> | <stat key="footprint.registries.cities.search_index.time_ms:load">~172ms</stat> / <stat key="footprint.registries.cities.search_index.memory_bytes:size">42.5MB</stat> | **<stat key="footprint.registries.cities.combined.time_ms:load">~743ms</stat> / <stat key="footprint.registries.cities.combined.memory_bytes:size">113.9MB</stat>** |
| **Total** | **<stat key="data.totals.records:int">297,335</stat>** | **<stat key="footprint.totals.dataset.time_ms:load">~354ms</stat> / <stat key="footprint.totals.dataset.memory_bytes:size">40.1MB</stat>** | **<stat key="footprint.totals.lookup_index.time_ms:load">~75ms</stat> / <stat key="footprint.totals.lookup_index.memory_bytes:size">6.1MB</stat>** | **<stat key="footprint.totals.filter_index.time_ms:load">~290ms</stat> / <stat key="footprint.totals.filter_index.memory_bytes:size">54.9MB</stat>** | **<stat key="footprint.totals.search_index.time_ms:load">~229ms</stat> / <stat key="footprint.totals.search_index.memory_bytes:size">56.5MB</stat>** | **<stat key="footprint.totals.combined.time_ms:load">~948ms</stat> / <stat key="footprint.totals.combined.memory_bytes:size">157.5MB</stat>** |

A registry also loads the datasets it references, without their indexes: a cold call on cities loads countries and subdivisions too.

> ⚠️ A [population threshold](#population-threshold) is the lever on memory: at <stat key="data.cities.threshold:int">15,000</stat>, cities drop from <stat key="data.cities.total:int">236,866</stat> to <stat key="data.cities.above_threshold:int">34,213</stat> and their memory from <stat key="footprint.registries.cities.combined.memory_bytes:size">113.9MB</stat> to <stat key="footprint.cities_threshold.memory_bytes:size">26.4MB</stat>.

### Search Benchmarks

`get()`, `lookup()` and `filter()` return in microseconds on warm caches.

| Registry | Latency (p50 / p95) | Queries | Accuracy (top 10) | Top Result |
|---|---|---|---|---|
| Countries | <stat key="bench.registries.countries.search.p50_ms:latency">1ms</stat> / <stat key="bench.registries.countries.search.p95_ms:latency">2.82ms</stat> | <stat key="bench.registries.countries.accuracy.queries:int">423</stat> | <stat key="bench.registries.countries.accuracy.success_pct:pct">100.0%</stat> | <stat key="bench.registries.countries.accuracy.top1_pct:pct">99.3%</stat> |
| Subdivisions | <stat key="bench.registries.subdivisions.search.p50_ms:latency">1.7ms</stat> / <stat key="bench.registries.subdivisions.search.p95_ms:latency">3.13ms</stat> | <stat key="bench.registries.subdivisions.accuracy.queries:int">7,003</stat> | <stat key="bench.registries.subdivisions.accuracy.success_pct:pct">97.6%</stat> | <stat key="bench.registries.subdivisions.accuracy.top1_pct:pct">93.4%</stat> |
| Cities | <stat key="bench.registries.cities.search.p50_ms:latency">4.55ms</stat> / <stat key="bench.registries.cities.search.p95_ms:latency">7.83ms</stat> | <stat key="bench.registries.cities.accuracy.queries:int">5,000</stat> | <stat key="bench.registries.cities.accuracy.success_pct:pct">98.7%</stat> | <stat key="bench.registries.cities.accuracy.top1_pct:pct">93.4%</stat> |

Accuracy is measured on names and aliases, each with at least one typo, of up to <stat key="bench.sample_size:int">5,000</stat> sampled records per registry; city queries add the admin1. A record sharing the sampled one's name (and a city's admin1) counts as a hit. Measured on <stat key="bench.host.cpu">12th Gen Intel(R) Core(TM) i9-12900KF</stat> with Python <stat key="bench.host.python">3.14.4</stat>.

## Concurrency

Registries are safe to share across threads, and a cold registry loads once even when several threads reach it together.

> ⚠️ `cities.set_population_threshold()` and `countries.set_include_historic()` apply to every thread: set them before sharing the registry, never while other threads query it.

### Batch searching

On free-threaded Python (3.14t), a thread pool searches in parallel:

```python
from concurrent.futures import ThreadPoolExecutor
import localis

# load once, before the threads start
localis.cities.force_cache()
with ThreadPoolExecutor() as pool:
    results = list(pool.map(localis.cities.search, queries))
```

On a standard build, use processes. Each worker loads its own copy of the data (up to <stat key="footprint.registries.cities.combined.memory_bytes:size">113.9MB</stat> for cities) and needs the parent's settings repeated:

```python
from concurrent.futures import ProcessPoolExecutor
import localis

def init_worker():
    localis.cities.set_population_threshold(15000)

def search_city(query):
    return localis.cities.search(query)

# workers import this module, so the pool only starts in the parent
if __name__ == "__main__":
    with ProcessPoolExecutor(initializer=init_worker) as pool:
        results = list(pool.map(search_city, queries, chunksize=500))
```

## Data Sources

Refreshed monthly from:

- **Countries**
  - **Canonical**: [ISO 3166-1](https://www.iso.org/iso-3166-country-codes.html) data via [Debian's iso-codes project](https://salsa.debian.org/iso-codes-team/iso-codes)
  - **Merged**: [ISO 3166-3](https://www.iso.org/iso-3166-country-codes.html) withdrawn/historic country codes, also via Debian's iso-codes project
  - **Merged**: [GeoNames](https://www.geonames.org/) `countryInfo.txt`, continents included
  - **Merged**: Additional country aliases queried from [Wikidata](https://www.wikidata.org/): English labels, alternative labels and short names
- **Subdivisions**
  - **Canonical**: [ISO 3166-2](https://www.iso.org/iso-3166-country-codes.html) data via [Debian's iso-codes project](https://salsa.debian.org/iso-codes-team/iso-codes)
  - **Merged**: [GeoNames](https://www.geonames.org/) `admin1CodesASCII.txt` and `admin2Codes.txt`
  - **Merged**: [Wikidata](https://www.wikidata.org/) (ISO 3166-2 code ↔ GeoNames id)
  - **Merged**: Additional subdivision aliases from GeoNames' `alternateNamesV2` dump (Latin-script names only, filtered by [Unicode CLDR](https://cldr.unicode.org/)'s official-language data per country)
- **Cities**
  - [GeoNames](https://www.geonames.org/) `cities500.txt` dataset
- **Macroregions**
  - [Unicode CLDR](https://cldr.unicode.org/) territory containment and English territory names
- **Currencies**
  - **Canonical**: [ISO 4217](https://www.iso.org/iso-4217-currency-codes.html) data via [Debian's iso-codes project](https://salsa.debian.org/iso-codes-team/iso-codes)
  - **Merged**: each country's legal tender from [Unicode CLDR](https://cldr.unicode.org/)'s currency data
- **Scripts**
  - **Canonical**: [ISO 15924](https://www.unicode.org/iso15924/) data via [Debian's iso-codes project](https://salsa.debian.org/iso-codes-team/iso-codes)
  - **Merged**: [Unicode CLDR](https://cldr.unicode.org/)'s English script names, as aliases
- **Languages**
  - **Canonical**: [ISO 639-3](https://iso639-3.sil.org/) data via [Debian's iso-codes project](https://salsa.debian.org/iso-codes-team/iso-codes)
  - **Merged**: [Unicode CLDR](https://cldr.unicode.org/)'s English language names as aliases, each language's scripts, and each country's official languages

Names ship in Latin script.

[`docs/methodology.md`](https://github.com/dstoffels/localis/blob/main/docs/methodology.md) documents how each dataset is built and validated, and where it's known to be wrong. [`unmerged_subdivisions.md`](https://github.com/dstoffels/localis/blob/main/docs/unmerged_subdivisions.md) lists every ISO subdivision without a GeoNames counterpart.

### Data licensing

The shipped data is modified from these sources and stays under their licenses: iso-codes (LGPL-2.1-or-later), GeoNames (CC BY 4.0), Unicode CLDR (Unicode License v3) and Wikidata (CC0). [`src/localis/data/NOTICE`](https://github.com/dstoffels/localis/blob/main/src/localis/data/NOTICE) attributes each source, and the license texts are in [`LICENSES/`](https://github.com/dstoffels/localis/tree/main/LICENSES) and the wheel's metadata. Keep both with the data if you redistribute it.

## License

- **Code**: MIT ([`LICENSE`](https://github.com/dstoffels/localis/blob/main/LICENSE))
- **Data**: its sources' licenses, listed under [Data licensing](#data-licensing)

The package's license expression is `MIT AND LGPL-2.1-or-later AND CC-BY-4.0 AND Unicode-3.0 AND CC0-1.0`.

## History

localis began with some database cleanup. I found myself writing mountains of bespoke code to parse inconsistent, dirty data while trying to reconcile pycountry, GeoNames and Wikidata to name a few (Google Places was not in the budget). When the pipeline was complete and the data finally cleaned, I realized this mountain of code could be useful for others who might need a reliable offline solution, so here we are! Over the past few years I've taken great care to build a robust, reliable dataset, wrapped in a simple, performant interface. I hope you find it useful and please don't hesitate to contribute or report any issues.

## Contributing

- [Pull requests welcome](https://github.com/dstoffels/localis): [`docs/dev.md`](https://github.com/dstoffels/localis/blob/main/docs/dev.md) covers setup, the dev tasks and the data pipeline
- [Report issues](https://github.com/dstoffels/localis/issues)

Support this project:
- [GitHub Sponsors](https://github.com/sponsors/dstoffels)
- [PayPal](https://www.paypal.biz/danOstoffels)

### Contributors
- [@SchubmannM](https://github.com/SchubmannM): performance work in [#8](https://github.com/dstoffels/localis/pull/8) that inspired localis's `array('I')` index storage and lazy-loaded registries