Metadata-Version: 2.4
Name: charex
Version: 0.2.4
Summary: A unicode and character set explorer.
Author-email: "Paul J. Iutzi" <pji@mac.com>
License-Expression: MIT
Project-URL: Homepage, https://charex.readthedocs.io/en/latest/index.html#
Project-URL: Documentation, https://charex.readthedocs.io/en/latest/index.html#
Project-URL: Source, https://github.com/pji/charex
Classifier: Programming Language :: Python :: 3
Classifier: Operating System :: OS Independent
Requires-Python: >=3.12
Description-Content-Type: text/x-rst
License-File: LICENSE
Requires-Dist: flask
Requires-Dist: blessed
Dynamic: license-file

######
charex
######

`charex` is a Unicode and character set explorer for understanding
issues with character set translation and Unicode normalization.


Why Did I Make This?
====================
I find the ambiguity of text data interesting. In memory it's all ones
and zeros. There is nothing inherent to the data that makes `0x20` mean
a space character, but we've mostly agreed that it does. That "mostly"
part is what's interesting to me, and it's where a lot of fun problems lie.


How Do I Use This?
==================
It's in PyPI, so you can install it with `pip`, as long as you are
using Python 3.12 or higher::

    pip install charex

`charex` has four modes of operation:

*   Direct command line invocation,
*   An interactive shell,
*   A graphical user interface (GUI),
*   An application programming interface (API).


Command Line
------------
To get help for direct invocation from the command line::

    $ charex -h


Interactive Shell
-----------------
To launch the interactive shell::

    $ charex

That will bring you to the `charex` shell::

    Welcome to the charex shell.
    Press ? for a list of comands.

    charex>

From here you can type `?` to see the list of available commands::

    Welcome to the charex shell.
    Press ? for a list of comands.

    charex> ?
    The following commands are available:

    *  bl: Show the list of blocks.
    *  cd: Decode the given address in all codecs.
    *  ce: Encode the given character in all codecs.
    *  cl: List registered character sets.
    *  clear: Clear the terminal.
    *  ct: Count denormalization results.
    *  dm: Build a denormalization map.
    *  dn: Perform denormalizations.
    *  dt: Display details for a code point.
    *  el: List the registered escape schemes.
    *  es: Escape a string using the given scheme.
    *  fl: List registered normalization forms.
    *  gui: Start the GUI.
    *  help: Display command list.
    *  mf: Create emoji sequence for a region's flag.
    *  nl: Perform normalizations.
    *  ns: Show the list of named sequences.
    *  pf: List characters with a given property value.
    *  sh: Run in an interactive shell.
    *  sv: Show the list of standardized variants.
    *  up: List the Unicode properties.
    *  uv: List the valid values for a Unicode property.
    *  xt: Exit the charex shell.
    *  zc: Show the list of emoji ZWJ sequence categories.
    *  zl: Show the list of emoji ZWJ sequences.

    For help on individual commands, use "help {command}".

    charex>

And then type `help` then a name of one of the commands to learn what
it does::

    charex> help dn
    usage: charex dn [-h] [-m MAXDEPTH] [-n NUMBER] [-r] [-s SEED] form base

    Denormalize a string.

    positional arguments:
      form                  The normalization form for the denormalization. Valid
                            options are: casefold, nfc, nfd, nfkc, nfkd.
      base                  The base normalized string.

    options:
      -h, --help            show this help message and exit
      -m MAXDEPTH, --maxdepth MAXDEPTH
                            Maximum number of reverse normalizations to use for
                            each character.
      -n NUMBER, --number NUMBER
                            Maximum number of results to return.
      -r, --random          Randomize the denormalization.
      -s SEED, --seed SEED  Seed the randomized denormalization.

    charex>


GUI
---
To launch the `charex` GUI::

    $ charex gui


API
---
To import `charex` into your Python script to get a summary of a
Unicode character::

    >>> import charex
    >>>
    >>>
    >>> value = 'a'
    >>> char = charex.Character(value)
    >>> print(char.summarize())
    a U+0061 (LATIN SMALL LETTER A)


What Version of Unicode Does This Support?
==========================================
Parts of `charex` rely on `unicodedata` in the Python Standard
Library. This limits `charex` to supporting the version supported
by the version of Python you are running. There may be a bit of a lag as
new Python versions are released, but as of this release `charex`
supports:

*   Python 3.12: Unicode 15.0
*   Python 3.13: Unicode 15.1
*   Python 3.14: Unicode 16.0


What happened to Unicode 14.0?
------------------------------
I have the dependencies I use to generate the documentation in the
development dependencies. To support Unicode 14.0, I had to support
Python 3.11. To support Python 3.11, I had to use an old version of
Sphinx that has some vulnerabilities. To clean that up, I had to drop
support for Python 3.11 and Unicode 14.0.

Is there a way I could have fixed it without dropping Python 3.11
support? Probably. It's is something I'll try to look into when I
get time. If it's a problem for anyone, I'll try to prioritize it.


What Is Left To Do?
===================
The following features are planned for the v0.2.5 or later releases:

*   Emoji combiner.
*   Basic doctests for all public classes and functions.
*   Web API.
*   Registration for character set codecs.
*   Allow unicode version switching.

The list of Unicode properties can be found here: `Index`_

The list of Unihan properties is here: `tr38`_

.. _`Index`: https://www.unicode.org/reports/tr44/tr44-28.html#Property_Index
.. _`tr38`: https://www.unicode.org/reports/tr38/tr38-31.html


Changes in v0.2.4
-----------------
The following are the changes in v0.2.4:

*   Updated dependencies.
*   Support for Unicode v16.0 for Python 3.14.
*   Added support for the InCB property to Unicode v15.1.
*   Removed official support for Python 3.11 to allow `sphinx` to be updated.
*   Added some checking to see if Unicode properties had been mapped.
*   Added `alias_property()` to the defined API.
*   Added zalgo (glitch) text escape.
*   Added the following commands:

    *   `bl`: Shows the Unicode character blocks.
    *   `mf`: Makes the flag for the given ISO 3166 region code,
    *   `zc`: Lists the emoji ZWJ sequence categories.


Changes in v0.2.3
-----------------
The following are the changes in v0.2.3:

*   Move dependency management to `poetry`.
*   Move package into /src directory.
*   Use `tox` for regression testing.
*   Support Unicode 15.0 for running under Python 3.12.

    *   Add Unicode 15.0 files.
    *   Added "kalternatetotalstrokes" property.
    *   Moved "kcihait" property source.
    *   Added "idna2008" property.

*   Support Unicode 15.1 for running under Python 3.13.

    *   Add Unicode 15.1 files.
    *   Remove seven Unihan properties.

        *   kHKSCS
        *   kIRGDaiKanwaZiten
        *   kKPS0
        *   kKPS1
        *   kKSC0
        *   kKSC1
        *   kRSKangXi

    *   Add six Unihan properties.

        *   kJapanese
        *   kMojiJoho
        *   kSMSZD2003Index
        *   kSMSZD2003Readings
        *   kVietnameseNumeric
        *   kZhuangNumeric

    *   Add "IDS_Unary_Operator" property.
    *   Add "ID_Compat_Math_Start" property.
    *   Add "ID_Compat_Math_Continue" property.
    *   Add "NFKC_Simple_Casefold" property.
    *   Check if multiple "kPrimaryNumeric" values are supported
        (see U+5146 and U+79ED).

*   Use Unicode version supported by Python version.

    *   Updated the path_map to handle different Unicode versions.
    *   Updated the prop_map to handle different Unicode versions.
    *   Can specify the Unicode version of a `FileCache`.
    *   Create a `FileCache` for the Unicode version supported by
        the running Python on launch.

*   Generate denormalizations for each version.
*   Return an AttributeError rather than KeyError when an attribute
    doesn't exist.


How Do I Run the Tests?
-----------------------
`charex` is using the `pytest` package for unit testing. It also
comes with a makefile that automates testing. So, to run the
tests:

*   Install `poetry`: `pip install poetry`
*   Install the development dependencies: `poetry install`
*   To run just the unit tests: `make test`
*   To run the full test suite: `make pre`


Common Problems
===============

`ModuleNotFoundError: No module name '_tkinter'` error
------------------------------------------------------
If you get the above error when running `charex` or its tests, it's
likely your Python install doesn't have `tkinter` linked. How you
fix it depends upon your Python install. If you are using Python 3.14
installed with `homebrew` on macOS, you can probably fix it with::

    brew install python-tk@3.14
