We briefly describe the use of cygenja. Basically, you register some filters, file extensions and actions before triggering the translation. In the Examples section, you can see cygenja in action as we detail its use to generate the CySparse library.
The Generator
class is the main class of the cygenja library. It also is the only
class you need to interact with. Its constructor is really simple:
from cygenja.generator import Generator
...
logger = ...
env = ...
engine = Generator('root_directory', env, logger, True)
You provide a root_directory
directory, a Jinja2 environment, a logger
(from the standard logging
library) and decide if warnings must raise Exceptions or not (default: False
). The first two arguments are mandatory while the last two are optional. You don’t have to provide
a logging engine. We describe the root directory a little further in The root directory, refer the reader to jinja2.Environment for more about
the env
argument and discuss the two last arguments
in the next corresponding subsections.
A logging engine can be used but is not mandatory. If you don’t want to log cygenja‘s behavior, simply pass None as the value of the logger argument in the constructor (this is the default). The logging engine is an object from Python’s logging library.
from cygenja.generator import Generator
import logging
logger = logging.getLogger('Logger name')
engine = Generator('root_directory', logger, True)
Three message logging helpers are provided:
def log_info(self, msg)
def log_warning(self, msg)
def log_error(self, msg)
Their names and signatures are quite self-explanatory.
Errors always trigger RuntimeErrors while warnings may or may not trigger RuntimeErrors. To raise exceptions on warnings, set the raise_exception_on_warning
to True
in the
constructor:
engine = Generator('root_directory', logger=logger, raise_exception_on_warning=True)
By default, raise_exception_on_warning
is set to False
.
There are only two types of patterns:
This is a general rule for the whole library. When you register an action though, you must provide a directory name, not a directory name pattern.
We encourage the reader to (re)read the specifications of these two libraries.
The root directory is really the main working directory: all file generations can only be done inside subdirectories of this directory.
This is so important that it is worth a warning:
Warning
File generations can only be done inside subdirectories of the root directory.
This directory is given as the first parameter of Generator
‘s constructor
and can be absolute or relative. At any moment, you can retrieve this directory as an absolute path:
engine = Generator('root_directory', ...)
absolute_root_directory = engine.root_directory()
Filters are simply Jinja2 filters. These filters are registered:
def my_jinja2_filter(filter_argument):
...
return filter_result
engine = Generator(...)
engine.register_filter('my_filter_name', my_jinja2_filter)
where 'my_filter_name'
is the name of the filter used inside your Jinja2 template files and my_jinja2_filter
is a reference to the actual filter.
The signature of register_filter
is:
register_filter(self, filter_name, filter_ref, force=False)
allowing you to register a new filter under an already existing filter name. If you keep force
set to False
, a warning is triggered each time you try to register a
new filter under an already existing filter name and this new filter is disregarded.
You also can register several filters at once with a dictionary of filters:
engine = Generator(...)
filters = { 'f1' : filter1,
'f2' : filter2}
engine.register_filters(filters, force=False)
At any time, you can list the registered filters:
engine = Generator(...)
print engine.filters_list()
This list also includes predefined Jinja2 filters (see builtin filter). If you only want the filters you registered, invoke:
engine.registered_filters_list()
cygenja uses a correspondance table between template files and generated files. This table defines a correspondance between file extensions. For instance, to have *.cpd templates generate *.pxd files:
engine = Generator(...)
engine.register_extension(`.cpd`, `.pxd`)
Again, we use a force
switch to force the redefinition of such a correspondance. By default, this switch is set to False
and if you try to redefine an association with a given template extension, you will
trigger a warning and this new correspondance will be disregarded.
You can use a dict
to register several extensions at once:
engine = Generator(...)
ext_correspondance = { '.cpd' : '.pxd',
'.cpx' : 'pyx'}
engine.register_extensions(ext_correspondance, force=False):
As with filters, you can retrieve the registered extensions:
engine.registered_extensions_list()
Files with extensions registered as template file extensions are systematically parsed, i.e. you cannot use these extensions for files that are not templates because cygenja will try to parse them. What about generated file extensions? Files with these extensions can peacefully coexist with generated files, i.e. existing files, regardless of their extensions, can coexist with generated files and will not be plagued by cyjenja. This means that you can safely delete files: only generated files will be deleted [1].
Note
Only generated files are deleted. You can thus safely delete files with cygenja.
Actions (defined in the GeneratorAction
class) are really the core concept of cygenja: an action correspond to a translation rule. This translation rule makes a correspondance between a subdirectory
and a file pattern and a user callback. Here is the signature of the register_action
method:
def register_action(self, relative_directory, file_pattern, action_function)
The relative_directory
argument holds the name of a relative directory from the root directory. The separator is OS dependent. For instance,
under linux, you can register the following:
engine = Generator(...)
def action_function(...):
...
return ...
engine.register_action('cysparse/sparse/utils', 'find*.cpy', action_function)
This means that all files corresponding to the 'find*.cpy'
fnmatch pattern inside the cysparse/sparse/utils
directory can be dealt with the action_function
.
Contrary to filters and file extensions, you cannot ask for a list of registered actions. But you can ask cygenja to perform a dry session: cygenja outputs what it would normaly do but without taking any action [2].
The action_function()
is a user-defined callback without argument. It returns a file suffix with a corresponding Jinja2
variables dict (this is a simple Python dict
). Let’s illustrate this by an example:
GENERAL_CONTEXT = {...}
INDEX_TYPES = ['INT32', 'INT64']
ELEMENT_TYPES = ['FLOAT32', 'FLOAT64']
def generate_following_index_and_type():
"""
"""
for index in INDEX_TYPES:
GENERAL_CONTEXT['index'] = index
for type in ELEMENT_TYPES:
GENERAL_CONTEXT['type'] = type
yield '_%s_%s' % (index, type), GENERAL_CONTEXT
The user-defined callback generate_following_index_and_type()
doesn’t take any input argument and returns the '_%s_%s'
suffix string together with the variables dict
GENERAL_CONTEXT
.
This function allows cygenja to create files with this suffix from any matching template file. The GENERAL_CONTEXT
is given to Jinja2 for the appropriate translation.
For instance, let’s use the ext_correspondance
extensions dict
discussed earlier (see File extensions):
ext_correspondance = { '.cpd' : '.pxd',
'.cpx' : 'pyx'}
Any template file with a .cpd
or .cpx
extension will be translated into a _index_type.pxd
or _index_type.pyx
file respectively. For instance, the template file my_template_code_file.cpd
will be translated to:
my_template_code_file_INT32_FLOAT32.cpd
my_template_code_file_INT32_FLOAT64.cpd
my_template_code_file_INT64_FLOAT32.cpd
my_template_code_file_INT64_FLOAT64.cpd
As this function is defined by the user, you have total control on what you want to generate or not. In our example, we redefine GENERAL_CONTEXT['index']
and GENERAL_CONTEXT['type']
for each index and element types.
We use generators (yield
) but you could return a list
if you prefer.
You could register incompatible actions, i.e. register competing actions that would translate a file in different ways. Our approach is to only use the first compatible action and to disregard all the other actions, regardless if they could be applied or not. So the order in which you register your actions is important. A file will be dealt with the first compatible action found. This is worth a warning:
Warning
A template is translated with the first compatible action found and only that action.
cygenja allows to define one default action that will be triggered when no other compatible action is found for a given template file that corresponds to a fnmatch pattern:
engine = Generator(...)
def default_action():
return ...
engine.register_default_action('*.*', default_action)
Be careful when defining a default action. This action is applied to all template files (corresponding to the fnmatch pattern) for which no compatible action is found. You might want to prefer declaring explicit actions than relying on this implicit default action. That said, if you have lots of default cases, this default action can be very convenient and avoid lots of unnecessary action declarations.
To generate the files from template files, there is only one method to invoke: generate(). Its signature is:
def generate(self, dir_pattern, file_pattern, action_ch='g',
recursively=False, force=False)
dir_pattern
is a glob
pattern taken from the root directory and it is only used for directories while file_pattern
is a fnmatch
pattern taken from all matching directories and is only used for files.
The action_ch
is a character that triggers different behaviours:
g
: Generate all files that match both directory and file patterns. This is the default behavior.d
: Same as g but with doing anything, i.e. dry run.c
: Same as g but erasing the generated files instead, i.e. clean.These actions can be done in a given directory or in all its corresponding subdirectories. To choose between these two options, use the recursively
switch. Finally, by default, files are only generated if they are
outdated, i.e. if they are older than the template they were originated from. You can force the generation with the force
switch.
Footnotes
[1] | The user is responsible to not to define a translation rule that overwrites any existing files. |
[2] | You also have access to the internal engine = Generator(...)
treemap = engine.registered_actions_treemap()
and thus you have access to all its methods. One interesting method is |