Metadata-Version: 2.5
Name: jazzy_fish
Version: 0.5.0
Summary: Generate sufficiently-large, unique, human-friendly identifiers, represented as English-language phrases.
Project-URL: Bug reports, https://github.com/MihaiBojin/jazzy-fish/issues/new
Project-URL: Documentation, https://github.com/MihaiBojin/jazzy-fish/blob/main/python/README.md
Project-URL: Source, https://github.com/MihaiBojin/jazzy-fish
Author-email: Mihai Bojin <557584+MihaiBojin@users.noreply.github.com>
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: decoder,encoder,human-friendly,identifiers,jazzyfish,unique
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Natural Language :: English
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Requires-Python: >=3.12
Provides-Extra: cli
Provides-Extra: dev
Requires-Dist: pre-commit==4.6.1; extra == 'dev'
Requires-Dist: pytest==9.1.1; extra == 'dev'
Requires-Dist: wheel-inspect==1.8.0; extra == 'dev'
Description-Content-Type: text/markdown

# Jazzy Fish - Sufficiently-large, unique, human-friendly identifiers

![Build Status](https://github.com/MihaiBojin/jazzy-fish/actions/workflows/cicd.yml/badge.svg)
[![PyPI version](https://badge.fury.io/py/jazzy-fish.svg)](https://badge.fury.io/py/jazzy-fish)
[![Python Versions](https://img.shields.io/pypi/pyversions/jazzy-fish.svg)](https://pypi.org/project/jazzy-fish/)
[![License](https://img.shields.io/github/license/MihaiBojin/jazzy-fish.svg)](LICENSE)

> ⚠️ **Warning:** This library is **in development** and not yet ready for production. Its API is not yet stable
> and might (and probably _will_) still change. Follow repository [issues](https://github.com/MihaiBojin/jazzy-fish/issues) for more information.

Jazzy Fish is a library that helps you generate a sufficient number of identifiers, with a human-friendly kick.

This is not a new idea and similar implementation can be found in various places
(i.e., GitHub new repository name [suggestions](https://github.com/new)).

Jazzy Fish is able to map a large integer solution space to unique keyphrases that can be used as human-friendly identifiers.
Each keyphrase also has a short abbreviated form that can be used as a compact identifier.

The implementation roughly works as follows:

- configure a `Generator` (details below)
- call `generator.next_id()`, which returns a unique integer value; values increase over time
  for a single configured machine id, but interleaving several machine ids means consecutive
  calls are not ordered
- call `encoder.encode(id)`, which returns a `KeyPhrase` with all three text forms
- call `encoder.decode(text)` to decode any text form to its integer

## Terminology

- `word part`: parts of speech used in the English language that can be used to form sentences/phrases: adverbs, verbs, adjectives, and nouns
- `dictionary`: a list of English words categorized by _word part_, which is used to generate _wordlists_
- `wordlist`: a list of *word part*s that can be combined to generate a large number of *key phrase*s
- `keyphrase`: a uniquely ordered sequence of words, each of a certain _word part_, which can be mapped to a unique integer identifier
  (e.g., `niftier engine`)
- `(keyphrase) abbreviations`: an ASCII representation of the keyphrase, which can be used
  as a non-numerical identifier (e.g., `nif-eng`); it is fixed-length when `min_phrase_size`
  equals the number of word lists, and variable below that
- `identifier`: a numerical (integer) value that can be used to represent unique entities (e.g., `12040320103821`)

## Quickstart

The project is published to <https://pypi.org/project/jazzy-fish/>.
Install it via:

```shell
pip install jazzy-fish

# or alternatively, directly from git
pip install "git+https://github.com/MihaiBojin/jazzy-fish@main#subdirectory=python"
```

The implementation roughly works as follows:

- configure a `Generator` (details below)
- call `generator.next_id()`, which returns a unique integer value; values increase over time
  for a single configured machine id, but interleaving several machine ids means consecutive
  calls are not ordered
- call `encoder.encode(id)`, which returns a `KeyPhrase` with all three text forms
- call `encoder.decode(text)` to decode any text form to its integer

## Identifier forms

`Generator.next_id()` produces the integer. `WordEncoder.encode(id)` returns a
`KeyPhrase` containing that integer and all three text forms. For `1234567` with
wordlist `012_80a1774`:

| Field | Value | Checks the wordlist tag when decoded |
| ----- | ----- | ----------------------------------- |
| `keyphrase` | `abjectly-abashed-moodier-gopher` | No |
| `abbr` | `abj-aba-moo-gop` | No |
| `verified_abbr` | `abj-aba-moo-gopmj` | Yes |

`encoder.decode(text)` accepts every form in the table.
`encoder.decode_abbr(text)` accepts either abbreviation form. A supplied tag must
match the configured wordlist; decoding rejects a mismatched tag.

The two trailing base-36 characters derive from the full wordlist name. They are
constant for that wordlist, so they do not check the identifier for typos. There
are 1,296 possible tags, so different wordlists can share one. Persist
`encoder.wordlist_name` alongside stored identifiers to select the correct
wordlist when decoding.

Full words contain no wordlist tag. For example,
`abjectly-abashed-abased-abbess` decodes to `2` under `012_80a1774` and `4` under
`024_e4d0f5e`. Both the keyphrase and the plain abbreviation rely on the caller
selecting the correct wordlist. If a custom wordlist makes a full phrase also
look like an abbreviation, `decode()` gives the full phrase precedence; use
`decode_abbr()` when the input is known to be an abbreviation.

`KeyPhrase` has four tuple fields in order: `id`, `abbr`, `keyphrase`, and
`verified_abbr`. Positional construction and unpacking require all four fields.

## Configuring a Generator

Integer IDs are constructed by combining 3 parts:

- a `timestamp`: relative to the UNIX epoch, or a custom epoch - to maximize the possible solution size;
  the resolution is one of `Resolution.MINUTE`, `Resolution.SECOND`, or `Resolution.MILLISECOND`
- a `machine id`: since it may be necessary to run multiple generators (i.e., in distributed systems), the solution domain can be partitioned by multiple 'machines'
- a `sequence id`: representing a number of identifiers that can be generated, all things being equal (e.g., same time, same machine)

Thus, the algorithm is configurable enough to split a solution domain (e.g., N potential word combinations, where N is a large integer) into smaller partitions, that can be reasoned about in terms of: `For how many years can IDs/word sequences be generated before the implementation needs to be changed?`

The idea behind this implementation is also inspired from Bitcoin's Improvement Proposal [39](https://github.com/bitcoin/bips/blob/master/bip-0039.mediawiki).

Note: The BIP39 implementation uses a single word list to convert 12 or 24 unique words out of a total of 2048 words into a very large integer that can be used to derive secret keys.

Jazzy Fish different from BIP39 in that it uses multiple word lists (specifically, adverbs, verbs, adjectives, and nouns) to generate word sequences that are similar to natural (English) language, with the assumption that sequences such as `yellow cat`, `hectic fish`, `dreadful elephant` (while somewhat nonsensical) are easy to memorize by humans used to combining word parts. So, the aim of this library is to choose sufficiently-large word lists that can generate sufficiently-large unique word sequences, for a reasonable duration (i.e., several years or more).

Another relevant detail of this algorithm, is its ability to map chosen word sequences to smaller prefixes that can be used to form constant-length identifiers.
While each sequence maps to an integer, remembering integers is hard for most humans. Thus, based on this implementation's assumption that humans can remember structured sentences, it selects the input wordlists in such a way that, for a given and pre-configured prefix length, there exists a single word that corresponds to that prefix.

For example, given a prefix length of size=1, `yellow cat` can be encoded to `yc` and then decoded back to the same two words. In this example, `yellow` is an adjective, and `cat` is a noun. There do not exist any other adjectives that start with `y`, nor nouns that start with `c` in our input word lists.

The reference implementation of the algorithm comes with a default wordlist of prefix 3, containing adverbs, verbs, adjectives, and nouns.

It can map the following solution domains:

- 2,178,560 unique combinations of `adjective noun`
- 2,740,628,480 unique combinations of `verb adjective noun`
- 1,205,876,531,200 unique combinations of `adverb verb adjective noun`

Two-word sequences may be impractical for sustained identifier generation, however, three word and four word sequences can sustain 87 and 38,238 years respectively at a rate of 1 identifier generated per second, using a single machine.

### Threads and generator ownership

| Class | Caller responsibility |
|---|---|
| `Generator` | Serialize access with single-thread ownership or an external lock |
| `ThreadSafeGenerator` | Share the instance across threads; `next_id()` takes an internal lock |

`Generator.next_id()` has no lock. The GIL alone does not make its sequence updates
atomic. Use `ThreadSafeGenerator` when the library should synchronize callers:

```python
from jazzy_fish.generator import Resolution, ThreadSafeGenerator

generator = ThreadSafeGenerator(
    epoch=1727740800,
    resolution=Resolution.MILLISECOND,
    machine_ids=[3],
    machine_id_bits=4,
    sequence_bits=12,
)
identifier = generator.next_id()  # Any application thread may call this instance.
```

Allocate machine IDs per application process. Threads in a process share the
same generator and its machine IDs. The internal lock protects sequence state;
thread identity does not select a machine ID. Processes producing IDs in the
same domain need disjoint machine-ID allocations and matching epoch, resolution,
and bit widths. Avoid overlapping generator state for the same allocation.

Stop callers before changing configuration or the `current_time` callable.
IDs increase within each machine's sequence; interleaving machine IDs or callers
does not promise completion order. Machine and sequence bits consume the
encoder's ID budget; choose them using the capacity helpers below.

A shared `ThreadSafeGenerator` serializes generation. Additional callers provide
safe concurrent access but do not parallelize its generation loop. The
[portable benchmark](../benchmarks/README.md) measures direct calls and shared
callers across Python versions using fresh temporary environments.
SDK-managed workers and bulk generation are tracked in
[issue #124](https://github.com/MihaiBojin/jazzy-fish/issues/124).

### How long a configuration lasts

The combination counts above are a solution space, not a lifetime. The generator packs the
timestamp, machine id, and sequence into one integer, so every machine or sequence bit shifts
the timestamp left and halves how long the identifiers stay encodable.

Against the default wordlist, epoch 2024-05-30, millisecond resolution:

| `machine_id_bits` | `sequence_bits` | Years | Exhausted |
| ----------------- | --------------- | ----- | --------- |
| 0                 | 0               | 38.24 | 2062-08-15 |
| 1                 | 1               | 9.56  | 2033-12-18 |
| 3                 | 1               | 2.39  | 2026-10-19 |
| 5                 | 5               | 0.04  | 2024-06-12 |

Do not use the UNIX epoch (0) with the default wordlist: milliseconds since 1970 passed its
1,205,876,531,200 combinations in 2008. Either pick a recent epoch, choose a larger wordlist,
or use a coarser resolution.
The same budget expressed as throughput, over a ten-year horizon from a 2024 epoch at
millisecond resolution:

| wordlist | bits available | generation ceiling |
| -------- | -------------- | ------------------ |
| `012_80a1774` (default) | 1 | 2,000 IDs/second |
| `024_e4d0f5e` | 4 | 16,000 IDs/second |
| `01234_011cf27` | 6 | 64,000 IDs/second |

If the default wordlists are unsuitable, they can be changed. Consult the [Generate wordlists](#generate-wordlists) section for details.

## Contents

This directory contains the following resources:

- [src/jazzy_fish](./src/jazzy_fish): Python code that generates unique identifiers and can encode/decode them to keyphrases;
  this is the main package a client implementation needs to generate jazzy-fish keyphrases.
- [src/jazzy_fish_tools](./src/jazzy_fish_tools): Utility that works with wordlists, cleaning up input words (removing invalid or inappropriate words) and generating various combinations that allow a user to create new input wordlists with different capabilities.

## Jazzy Fish Tools

The library also provides tooling that can generate all combinations of wordlists, abbreviations of a given length,
and character positions chosen for the abbreviation.

```shell
generate-wordlists $PATH_TO_REPO/dictionary/5
```

These can help users infer the best choice depending on their use-case.

You may replace the wordlist, with a directory of your choosing, as long as it contains the same file structure - four files named after the relevant four parts in the English language:

- adverb.txt
- adjective.txt
- verb.txt
- noun.txt

After running this command, you can examine all outputs in the `out/processed` directory and you can see a high-level comparison by running `scripts/show-stats.bash`.

## Developers

Unless you are interested in contributing to this code (or are curious about this library's development processes), you can stop reading here.

The tests, Docker image scripts and release scripts require
[releasetools/cli](https://github.com/releasetools/cli) v0.4.0 or newer:

```shell
brew install releasetools/tap/releasetools-cli
rt version
```

CI installs v0.4.0 through the CLI's GitHub Action. A clean Docker build uses
the highest release tag on HEAD, without its `v` prefix, or the short commit
SHA when no release tag exists. A dirty build uses the SHA with `-dirty`.

### Publishing

#### GitHub-based version publishing

Releasing is merging a version bump. Nothing is tagged or published by hand.

```shell
# In a release branch: set the new version and refresh the lock file
#   (uv.lock records the project version, so CI's `uv sync --locked` fails without this)
uv lock
```

Open a pull request with that change. Merging it runs `cicd.yml`, a single pipeline of four
jobs:

1. **`lint-test`** -- linters and tests across every supported Python version.
2. **`tag`** -- reads `version` from `pyproject.toml` and tags the commit `v<version>`, unless that tag
   already exists. It needs `lint-test`, so nothing is tagged until the tests pass: a bad tag is far
   more awkward to retract than a failed build is to fix.
3. **`publish`** -- builds the tag, signs a [PEP 740](https://peps.python.org/pep-0740/) attestation
   for each dist, uploads both to test.PyPI and PyPI, then verifies PyPI serves the attestations back.
4. **`github-release`** -- turns the tag into a [release](https://github.com/MihaiBojin/jazzy-fish/releases),
   with notes generated from the pull requests merged since the previous one. The notes are only as
   informative as the pull request titles they are built from.

Merges that do not change the version find their tag already present, and `publish` is skipped. Pull
requests run `lint-test` only.

`publish` re-runs `rt git::assert_tag_version "$(uv version --short)"` before building and refuses to publish if the tag
and `pyproject.toml` disagree.

The three steps live in one workflow deliberately. Splitting them meant a workflow pushing a tag and
another watching for it, which does not work directly: GitHub suppresses events made with
`GITHUB_TOKEN`, so a pushed tag raises no `push` event, and bridging that gap needs either a
long-lived personal access token or a `workflow_dispatch` hop. Job dependencies express the same
ordering with none of that.

Note the filename is load-bearing: PyPI and test.PyPI each register a trusted publisher against
`cicd.yml`, and OIDC uploads stop working the moment it disagrees.

The workflow authenticates with [Trusted Publishing](https://docs.pypi.org/trusted-publishers/), so it
holds no API tokens. PyPI and test.PyPI each exchange the workflow's OIDC identity for a short-lived
upload token, which requires `id-token: write` on the publishing job and a trusted publisher registered
on **both** indexes, pointing at `cicd.yml` in this repository.

### Generate wordlists

The [jazzy_fish_tools](src/jazzy_fish_tools) package contains code that can process dictionaries and generate all combinations of wordlists, abbreviations of a given length, and character positions chosen for the abbreviation. These can help users infer the best choice depending on their use-case.

Install the package to use both CLI commands:

```shell
pip install jazzy-fish
```

Then, call [generate-wordlists PATH_TO_DICTIONARY_DIR](src/jazzy_fish_tools/generate_wordlists.py) to generate all possible combinations.
The resulting wordlists, checksums, and statistics are stored in `out/processed`.
Both commands use the Python standard library. The `[cli]` extra is accepted for installation compatibility and installs no additional dependencies.

If you want to generate wordlists using a dictionary of your choosing, use the [clean-dictionary PATH_TO_DICTIONARY_DIR](src/jazzy_fish_tools/clean_dictionary.py)
script to sanitize the inputs (in-place). Consult one of the included dictionaries ([dictionary/](dictionary/)) to determine the required file structure.
