Tokenizing related methods
===========================

.. note::

   While this documentation is extracted from a submodule,
   as shown below, all information is available from
   the top level, i.e. you can (should) use::

      from token_utils import tokenize, untokenize

   since this will always be guaranteed to work, even
   if refactoring occurs.


API extracted by Sphinx
--------------------------------------

.. automodule:: token_utils.tokenizing
   :members:
