pyTenjin User's Guide

release: 0.9.0
last update: $Date$

Table of Contents:

Overview

Tenjin is a very fast and full-featured template engine implemented in pure Python.

ATTENTION: pyTenjin will change some featrues in the next release. See Planned Changes in the Next Relase section for details.

Features


Install

pyTenjin supports Python 2.3 or later. Python 3 is also supported.

You can install Tenjin by easy_install command.

$ sudo easy_install Tenjin

Or download Tenjin-X.X.X.tar.gz, extract it, and type 'python setup.py install'.

$ wget http://downloads.sourceforge.net/project/tenjin/pytenjin/X.X.X/Tenjin-X.X.X.tar.gz
$ tar xzf Tenjin-X.X.X.tar.gz
$ cd Tenjin-X.X.X/
$ sudo python setup.py install

Benchmark

Tenjin package contains benchmark program.

MacOS X 10.6 Snow Leopard, Intel CoreDuo2 2GHz, Memory 2GB
$ cd pytenjin-X.X.X/benchmark
$ python -V
Python 2.5.5
$ python bench.py -q -n 10000
*** ntimes=10000
compiling bench_cheetah.tmpl ... Compiling bench_cheetah.tmpl -> bench_cheetah.py (backup bench_cheetah.py.bak)
*** loading context data (file=bench_context.py)...
*** start benchmark
*** ntimes=10000
                                    utime     stime     total      real
tenjin                             3.8700    0.0000    3.8700    3.8858
tenjin-create                      4.9600    0.5600    5.5200    5.5457
tenjin-str                         2.4900    0.0000    2.4900    2.4967
tenjin-webext                      2.3500    0.0000    2.3500    2.3703
django                            70.5100    0.0500   70.5600   70.8199
django-create                     83.7900    0.3500   84.1400   84.5050
cheetah                           15.6900    0.0200   15.7100   15.7504
cheetah-create                    16.6700    0.0100   16.6800   16.7545
kid                              287.4500    0.2900  287.7400  289.1525
kid-create                       288.6500    0.4800  289.1300  290.7973
genshi                           208.2000    0.2400  208.4400  209.4677
genshi-create                    452.4900    1.5200  454.0100  456.4957
mako                               7.0500    0.0400    7.0900    7.1348
mako-create                        9.7300    0.8100   10.5400   10.5842
mako-nocache                     110.9300    0.6200  111.5500  112.1807
templetor                         47.5300    0.0700   47.6000   47.8334
templetor-create                 362.8200    2.0100  364.8300  367.4523
jinja2                             7.6800    0.0600    7.7400    8.1972
jinja2-create                    120.5900    0.6200  121.2100  122.0719

Versions:

This shows the followings.



Basic Examples

This section describes basics of Tenjin.

Render Template

Notation:

<?py ... ?>
Python statements
${...}
Python expression (with HTML escape)
#{...}
Python expression (without HTML escape)
views/page.pyhtml: html template
<h2>${title}</h2>
<table>
<?py i = 0 ?>
<?py for item in items: ?>
<?py     i += 1 ?>
<?py     klass = i % 2 and 'odd' or 'even' ?> 
  <tr class="#{klass}">
    <td>${item}</td>
  </tr>
<?py #endfor ?>
</table>
main.py: main program
## import modules and helper functions
import tenjin
from tenjin.helpers import *

## context data
context = {
    'title': 'Tenjin Example',
    'items': ['<AAA>', 'B&B', '"CCC"'],
}

## cleate engine object
engine = tenjin.Engine(path=['views'])

## render template with context data
html = engine.render('page.pyhtml', context)
print(html)
result
$ python main.py
<h2>Tenjin Example</h2>
<table>
  <tr class="odd">
    <td>&lt;AAA&gt;</td>
  </tr>
  <tr class="even">
    <td>B&amp;B</td>
  </tr>
  <tr class="odd">
    <td>&quot;CCC&quot;</td>
  </tr>
</table>

Tips: you can check template syntax by 'pytenjin -z'.

Syntax check of template files
$ pytenjin -z views/*.pyhtml
views/page.pyhtml - ok.

Template Syntax

You can write any Python statements or expression in yor template file, but there are several restrictions.

NG examples
## NG because 'for' is not closed by '#endfor'
<ul>
<?py for item in items: ?>
  <li>${item}</li>
<?py #end ?>
</ul>

## NG because indent is 2 spaces
<?py if x > 0: ?>
<?py   if y > 0: ?>
<?py     if z > 0: ?>
<?py     #endif ?>
<?py   #endif ?>
<?py #endif ?>

## NG because conditional expression of 'if' is not in a line
<?py if re.search(r'^\[(\w+)\]',?>
<?py              line, re.M): ?>
<p>matched.</p>
<?py #endif ?>

## NG because there is a comment after ':' and '#endfor'
<ul>
<?py if item in items:   ## beginning of loop ?>
  <li>${item}</li>
<?py #endfor             ## end of loop ?>
</ul>

You may feel that indent restriction of Tenjin is strong and not convenient. If you feel so, see Flexible Indentation section.


Show Converted Source Code

pyTenjin converts template files into Python script and executes it. Compiled Python script is saved into cache file automatically. Cache file is saved with Python's marshal format (= bytecode format).

Show cached file
$ ls views/page.pyhtml*
views/page.pyhtml       views/page.pyhtml.cache
$ file views/page.pyhtml.cache
views/page.pyhtml.cache: data

You can get converted script by 'pytenjin -s'.

Show Python code
$ pytenjin -s views/page.pyhtml
_buf = []; _buf.extend(('''<h2>''', escape(to_str(title)), '''</h2>
<table>\n''', ));
i = 0
for item in items:
    i += 1
    klass = i % 2 and 'odd' or 'even'
    _buf.extend(('''  <tr class="''', to_str(klass), '''">
    <td>''', escape(to_str(item)), '''</td>
  </tr>\n''', ));
#endfor
_buf.extend(('''</table>\n''', ));
print(''.join(_buf))

If you specify '-sb' instead of '-s', neither preamble (= '_buf = [];') nor postamble (= 'print "".join(_buf)') are printed. See Retrieve Embedded Code section for more information.

Or:

How to convert template file into Python script
import tenjin
template = tenjin.Template('views/page.pyhtml')
print(template.script)

### or:
## engine = tenjin.Engine()
## print(engine.get_template('page.pyhtml').script)

Layout Template

Layout template will help you to use common HTML design for all pages.

views/_layout.pyhtml
<!DOCTYPE>
<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
    <title>${title}</title>
  </head>
  <body>
#{_content}
  </body>
</html>
main.py
## import modules and helper functions
import tenjin
from tenjin.helpers import *

## context data
context = {
    'title': 'Tenjin Example',
    'items': ['<AAA>', 'B&B', '"CCC"'],
}

## cleate engine object
engine = tenjin.Engine(path=['views'], layout='_layout.pyhtml')

## render template with context data
html = engine.render('page.pyhtml', context)
print(html)
Result
$ python main.py
<!DOCTYPE>
<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
    <title>Tenjin Example</title>
  </head>
  <body>
<h2>Tenjin Example</h2>
<table>
  <tr class="odd">
    <td>&lt;AAA&gt;</td>
  </tr>
  <tr class="even">
    <td>B&amp;B</td>
  </tr>
  <tr class="odd">
    <td>&quot;CCC&quot;</td>
  </tr>
</table>

  </body>
</html>

You can specify other layout template file with render() method.

## use other layout template file
engine.render('page.pyhtml', context, layout='_other_layout.pyhtml')

## don't use layout template file
engine.render('page.pyhtml', context, layout=False)

Tenjin supports nested layout template. See Nested Layout Template section for details.


Context Variables

'_context' variable contains context dictionary which is passed from main program to template file.

Using '_context' dictionary, you can pass any data from template file to main program and layout template file.

views/_layout.pyhtml
<!DOCTYPE>
<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
    <title>${page_title}</title>
  </head>
  <body>
#{_content}
  </body>
</html>
views/page.pyhtml: pass page title date from template to layout template
<?py _context['page_title'] = 'Tenjin: Layout Template Example' ?>
<h2>${title}</h2>
<table>
<?py i = 0 ?>
<?py for item in items: ?>
<?py     i += 1 ?>
<?py     klass = i % 2 and 'odd' or 'even' ?> 
  <tr class="#{klass}">
    <td>${item}</td>
  </tr>
<?py #endfor ?>
</table>
Result
$ python main.py
<!DOCTYPE>
<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
    <title>Tenjin: Layout Template Example</title>
  </head>
  <body>
<h2>Tenjin Example</h2>
<table>
  <tr class="odd">
    <td>&lt;AAA&gt;</td>
  </tr>
  <tr class="even">
    <td>B&amp;B</td>
  </tr>
  <tr class="odd">
    <td>&quot;CCC&quot;</td>
  </tr>
</table>

  </body>
</html>


Template Arguments

It is recommended to declare context variables in your template files for readability reason.

views/page.pyhtml
<?py #@ARGS title, items ?>
<?py _context['page_title'] = 'Tenjin: Layout Template Example' ?>
<h2>${title}</h2>
<table>
<?py i = 0 ?>
<?py for item in items: ?>
<?py     i += 1 ?>
<?py     klass = i % 2 and 'odd' or 'even' ?> 
  <tr class="#{klass}">
    <td>${item}</td>
  </tr>
<?py #endfor ?>
</table>
Converted Python script
$ pytenjin -s views/page.pyhtml
_buf = []
title = _context.get('title'); items = _context.get('items'); 
_context['page_title'] = 'Tenjin: Layout Template Example'
_buf.extend(('''<h2>''', escape(to_str(title)), '''</h2>
<table>\n''', ));
i = 0
for item in items:
    i += 1
    klass = i % 2 and 'odd' or 'even'
    _buf.extend(('''  <tr class="''', to_str(klass), '''">
    <td>''', escape(to_str(item)), '''</td>
  </tr>\n''', ));
#endfor
_buf.extend(('''</table>\n''', ));
print(''.join(_buf))
views/page.pyhtml
<?py #@ARGS _content, page_title ?>
<!DOCTYPE>
<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
    <title>${page_title}</title>
  </head>
  <body>
#{_content}
  </body>
</html>
Converted Python script
$ pytenjin -s views/_layout.pyhtml
_buf = []
_content = _context.get('_content'); page_title = _context.get('page_title'); 
_buf.extend(('''<!DOCTYPE>
<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
    <title>''', escape(to_str(page_title)), '''</title>
  </head>
  <body>
''', to_str(_content), '''
  </body>
</html>\n''', ));
print(''.join(_buf))

Tips: If you want to specify default value of context variable, try _context.get('varname', defaultvalue).

<?py
## if username is not specified, use 'World' as default.
username = _context.get('username', 'World')
?>
<p>Hello ${username}</p>

Include Partial Template

You can include other template files by include() helper function.

include(template-name, **kwargs)
Include other template. template-name can be file name or template short name. kwargs is passed to template as local variables.
views/_layout.pyhtml
<?py #@ARGS _content, page_title ?>
<!DOCTYPE>
<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
    <title>${page_title}</title>
  </head>
  <body>
<?py include('_header.pyhtml', title=page_title) ?>
#{_content}
<?py include('_footer.pyhtml') ?>
  </body>
</html>
views/_header.pyhtml
<?py #@ARGS title ?>
<div class="header">
  <h1>${title}</h1>
</div>
views/_footer.pyhtml
<?py #@ARGS ?>
<address>
  copyright(c) 2010 kuwata-lab.com all rights reserved
</address>
Result
$ python main.py
<!DOCTYPE>
<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
    <title>Tenjin: Layout Template Example</title>
  </head>
  <body>
<div class="header">
  <h1>Tenjin: Layout Template Example</h1>
</div>
<h2>Tenjin Example</h2>
<table>
  <tr class="odd">
    <td>&lt;AAA&gt;</td>
  </tr>
  <tr class="even">
    <td>B&amp;B</td>
  </tr>
  <tr class="odd">
    <td>&quot;CCC&quot;</td>
  </tr>
</table>

<address>
  copyright(c) 2010 kuwata-lab.com all rights reserved
</address>
  </body>
</html>


Template Short Name

If you specify template postfix, you can use template short name such as ':views/page' instead of 'views/page.pyhtml'. Notice that template short name should start with ':'.

main.py
## import modules and helper functions
import tenjin
from tenjin.helpers import *

## context data
context = {
    'title': 'Tenjin Example',
    'items': ['<AAA>', 'B&B', '"CCC"'],
}

## cleate engine object
engine = tenjin.Engine(path=['views'], postfix='.pyhtml', layout=':_layout')

## render template with context data
html = engine.render(':page', context)
print(html)
views/_layout.pyhtml
<?py #@ARGS _content, page_title ?>
<!DOCTYPE>
<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
    <title>${page_title}</title>
  </head>
  <body>
<?py include(':_header') ?>
#{_content}
<?py include(':_footer') ?>
  </body>
</html>

Template Encoding

If you want to specify encoding name of template file, add magic comment at 1st line in template file.

Magic comment example
<?py # -*- coding: utf-8 -*- ?>
<html>
  ...

Python 2.x

Tenjin provides two approaches for encoding in Python 2.x.

(A) Binary-based approach

Tenjin handles templates as binary file. For example, text "foo" will be converted into _buf.exnted(('foo', )). In this approach, unicode object should be encoded into binary(=str) by to_str().

to_str = tenjin.generate_tostrfunc(encode='utf-8')
   # ex. to_str(u'\u65e5\u672c\u8a9e') => '\xe6\x97\xa5\xe6\x9c\xac\xe8\xaa\x9e'
engine = tenjin.Engine()

If you have some troubles about encoding in this approach, add magic comment into your template file.

<?py # -*- coding: utf-8 -*- ?>
<h1>Hello</h1>
(B) Unicode-based approach

You can hanlde templates as unicode string with encoding option for tenjin.Engine(). For example, text "foo" will be converted into _buf.extend((u'foo', )). In this approach, binary(=str) should be decoded into unicode in to_str().

to_str = tenjin.generate_tostrfunc(decode='utf-8')
   # ex. to_str('\xe6\x97\xa5\xe6\x9c\xac\xe8\xaa\x9e')  #=> u'\u65e5\u672c\u8a9e'
engine = tenjin.Engine(encoding='utf-8')
html = engine.render('index.pyhtml')
if isinstance(html, unicode): html = html.encode('utf-8')
print(html)

You should not add encoding declaration in your templates, or you will get the following error.

SyntaxError: encoding declaration in Unicode string

In Python 2.x, binary-based approach is faster than unicode-based approach. If you don't have any special reason, it is recommend to be with binary-based approach.


Python 3.0

In Python 3.0, string is treated as unicode object. So Tenjin handles all templates in string base (= unicode base), and bytes data is converted into str by to_str() function. If you don't specify any encoding, Tenjin uses 'utf-8' encoding as default.

to_str = tenjin.generate_tostrfunc('utf-8')  ## decode bytes into unicode
engine = tenjin.Engine(encoding='utf-8')     ## handle template as unicode


Helper Function

Tenjin provides some modules for helper functions.

tenjin.helpers module

Module tenjin.helpers provides basic helper functions.

to_str(value)
Converts value into string. None is converted into empty string instead of "None". Unicode object is encoded into string with 'utf-8' encoding name. If you want to use other encoding name, use generate_tostr_func().
>>> from tenjin.helpers import to_str
>>> to_str(123)
'123'
>>> to_str(None)       # None is converted into empty string
''
>>> to_str(u'日本語')  # unicode object is encoded into str
'\xc3\xa6\xc2\x97\xc2\xa5\xc3\xa6\xc2\x9c\xc2\xac\xc3\xa8\xc2\xaa\xc2\x9e'
escape(str)
Escape HTML special characters.
>>> from tenjin.helpers import escape
>>> escape('< > & "')
'&lt; &gt; &amp; &quot;'
generate_tostrfunc(encode=encoding, decode=encoding)
Generate to_str() function with enoding name.
>>> from tenjin.helpers import generate_tostrfunc
>>> ## generate to_str() function which encodes unicode into binary(=str).
>>> to_str = generate_tostrfunc(encode='utf-8')
>>> to_str(u'hoge')
'hoge'
>>> ## generate to_str() function which decodes binary(=str) into unicode.
>>> to_str = generate_tostrfunc(decode='utf-8')
>>> to_str('hoge')
u'hoge'
echo(value)
Add value string into _buf. This is similar to echo() function of PHP.
## add _content into here
<?py echo(_content) ?>
## this is same as #{...}
#{_content}
not_cached(key, lifetime=0)
If fragment is expired or not cached, start caching fragment with key and lifetime (seconds), and returns True. If fragment is already cached with key, returns False. See Fragment Cache for details.
echo_cached()
Echo cached fragment. If caching is started by not_cached, stop and cache it. This function should be used with not_cached(). See Fragment Cache for details.
start_capture(name), stop_capture()
Start capturing with specified name. See Capturing section for details.
captured_as(name)
Return True if captured with name. See Capturing section for details.
_P(value), _p(value)
Helper method for preprocessing. See Preprocessing section for details.

tenjin.helpers.html module

Module tenjin.helpers.html provides HTML specific helper functions.

escape(str), escape_xml(str)
Escapes HTML special characters. Same as tejin.helpers.escape().
checked(value)
Returns ' checked="checked"' if value is true value, else returns empty string.
>>> from tenjin.helpers.html import *
>>> checked(1+1==2)
' checked="checked"'
>>> checked(1+1==3)
''
selected(value)
Returns ' selected="selected"' if value is true value, else returns empty string.
>>> from tenjin.helpers.html import *
>>> selected(1+1==2)
' selected="selected"'
>>> selected(1+1==3)
''
disabled(value)
Returns ' disabled="disabled"' if value is true value, else returns empty string.
>>> from tenjin.helpers.html import *
>>> disabled(1+1==2)
' disabled="disabled"'
>>> disabled(1+1==3)
''
new_cycle(*values)
Generates cycle object.
>>> from tenjin.helpers.html import *
>>> cycle = new_cycle('odd', 'even')
>>> cycle()
'odd'
>>> cycle()
'even'
>>> cycle()
'odd'
>>> cycle()
'even'
nl2br(str)
Replaces "\n" into "<br />\n".
>>> from tenjin.helpers.html import *
>>> nl2br("foo\nbar\nbaz\n")
'foo<br />\nbar<br />\nbaz<br />\n'
text2html(str)
(experimental) Escapes xml characters and replace "\n" into "<br />\n".
>>> from tenjin.helpers.html import *
>>> text2html('<AAA>\nB&B\n"CCC"\n')
'&lt;AAA&gt;<br />\nB&amp;B<br />\n&quot;CCC&quot;<br />\n'
tagattr(name, expr, value=None, escape=True)
(experimental) Returns ' name="value"' if expr is true value, else '' (empty string). If value is not specified, expr is used as value instead.
>>> from tenjin.helpers.html import *
>>> tagattr('name', 'account')
' name="account"'
>>> tagattr('name', None)
''
>>> tagattr('checked', True, 'checked')
' checked="checked"'
tagattrs(**kwargs)
(experimental) Builds html tag attribtes.
>>> from tenjin.helpers.html import *
>>> tagattrs(klass='main', size=20)
' class="main" size="20"'
>>> tagattrs(klass='', size=0)
''
nv(name, value, sep=None, **kwargs)
(experimental) Builds name and value attributes.
>>> from tenjin.helpers.html import *
>>> nv('rank', 'A')
'name="rank" value="A"'
>>> nv('rank', 'A', '-')
'name="rank" value="A" id="rank-A"'
>>> nv('rank', 'A', '-', checked=True)
'name="rank" value="A" id="rank-A" checked="checked"'
>>> nv('rank', 'A', '-', klass='error', style='color:red')
'name="rank" value="A" id="rank-A" class="error" style="color:red"'


Planned Changes in the Next Release (1.0.0)

This section describes planned changes in the next release (1.0.0).

(*1)
In Tenjin, string concatenation is not a bottleneck.


Advanced Features

Nested Layout Template

It is able to nest several layout template files.

views/_site_layout.pyhtml
<?py #@ARGS _content ?>
<html>
  <body>
#{_content}
  </body>
</html>
views/_blog_layout.pyhtml
<?py #@ARGS _content, title ?>
<?py _context['_layout'] = '_site_layout.pyhtml' ?>
<h2>${title}</h2>
<!-- content -->
#{_content}
<!-- /content -->
views/blog_post.pyhtml
<?py #@ARGS post_content ?>
<?py _context['_layout'] = '_blog_layout.pyhtml' ?>
<div class="article">
#{text2html(post_content)}
</div>
main.py
import tenjin
from tenjin.helpers import *
from tenjin.helpers.html import text2html
engine = tenjin.Engine(path=['views'])
context = {
    'title': 'Blog Post Test',
    'post_content': "Foo\nBar\nBaz",
}
html = engine.render('blog_post.pyhtml', context)
print(html)
Result
$ python main.py
<html>
  <body>
<h2>Blog Post Test</h2>
<!-- content -->
<div class="article">
Foo<br />
Bar<br />
Baz
</div>

<!-- /content -->

  </body>
</html>


Capturing

It is able to capture parital of template. It can be poor-man's alternative of Django's template-inheritance.

views/blog-post.pyhtml
<?py #@ARGS blog_post, recent_posts ?>
<h2>#{blog_post['title']}</h2>
<div class="blog-post">
#{text2html(blog_post['content'])}
</div>

<?py start_capture('sidebar') ?>
<h3>Recent Posts</h3>
<ul>
<?py for post in recent_posts: ?> 
  <a href="/blog/#{post['id']}">${post['title']}</a>
<?py #endfor ?>
</ul>
<?py stop_capture() ?>
views/_layout.pyhtml
<html>
  <body>
    <div id="header-part">
<?py if not captured_as('header'): ?>
      <h1>My Great Blog</h1>
<?py #endif ?>
    </div>
    <div id="main-content">
#{_content}
    </div>
    <div id="sidebar-part">
<?py if not captured_as('sidebar'): ?>
      <h3>Links</h3>
      <ul>
        <a href="http://google.com/">Google</a>
        <a href="http://yahoo.com/">Yahoo!</a>
      </ul>
<?py #endif ?>
    </div>
  </body>
</html>
main.py
## context data
blog_post = {
  'title': 'Tenjin is Great',
  'content': """
Tenjin has great features.
- Very Fast
- Full Featured
- Easy to Use
"""[1:]
}
recent_posts = [
  {'id': 1, 'title': 'Tenjin is Fast' },
  {'id': 2, 'title': 'Tenjin is Full-Featured' },
  {'id': 3, 'title': 'Tenjin is Easy-to-Use' },
]
context = {
  'blog_post': blog_post,
  'recent_posts': recent_posts,
}

## render template
import tenjin
from tenjin.helpers import *
from tenjin.helpers.html import text2html
engine = tenjin.Engine(path=['views'], layout='_layout.pyhtml')
html = engine.render('blog-post.pyhtml', context)
print(html)

The result shows that captured string (with name 'sidebar') overwrites layout template content.

Result
$ python main.py
<html>
  <body>
    <div id="header-part">
      <h1>My Great Blog</h1>
    </div>
    <div id="main-content">
<h2>Tenjin is Great</h2>
<div class="blog-post">
Tenjin has great features.<br />
- Very Fast<br />
- Full Featured<br />
- Easy to Use<br />

</div>


    </div>
    <div id="sidebar-part">
<h3>Recent Posts</h3>
<ul>
  <a href="/blog/1">Tenjin is Fast</a>
  <a href="/blog/2">Tenjin is Full-Featured</a>
  <a href="/blog/3">Tenjin is Easy-to-Use</a>
</ul>
    </div>
  </body>
</html>


Template Cache

Tenjin converts template file into Python script and save it as cache file. By default, it is saved as template-filename + '.cache' in bytecode format. You can change this behaviour by setting tenjin.Engine.cache or passing cache object to tenjin.Engine object.

For example, if you want to cache template object but want not to create '*.cache' file, use tenjin.MemoryCacheStorage object.

example to change template caching
import tenjin
from tenjin.helpers import *

## change to store template cache into memory instead of file system
tenjin.Engine.cache = tenjin.MemoryCacheStorage()
engine = tenjin.Engine()

## or
engine = tenjin.Engine(cache=tenjin.MemoryCacheStorage())

Fragment Cache

You can cache a certain part of HTML to improve performance. This is called as Fragment Cache.

views/items.pyhtml
<?py #@ARGS get_items ?>
<div>
<?py # fragment cache with key ('items/1') and lifetime (60sec) ?>
<?py if not_cached('items/1', 60): ?>
  <ul>
<?py     for item in get_items(): ?>
    <li>${item}</li>
<?py     #endfor ?>
  </ul>
<?py #endif ?>
<?py echo_cached() ?>
</div>

Tenjin stores fragments caches into memory by default. If you want to change or customize cache storage, see the following example.

main.py
import os, tenjin
from tenjin.helpers import *

## create key-value store object
if not os.path.isdir('cache.d'): os.mkdir('cache.d')
kv_store = tenjin.FileBaseStore('cache.d')      # file based

## set key-value store into tenjin.helpers.fagment_cache object
tenjin.helpers.fragment_cache.store = kv_store

## context data
## (it is strongly recommended to create function object
##  to provide pull-style context data)
def get_items():   # called only when cache is expired
    return ['AAA', 'BBB', 'CCC']
context = {'get_items': get_items}

## render html
engine = tenjin.Engine(path=['views'])
html = engine.render('items.pyhtml', context)
print(html)
Result
$ python main.py
<div>
  <ul>
    <li>AAA</li>
    <li>BBB</li>
    <li>CCC</li>
  </ul>
</div>

You'll find that HTML fragment is cached into cache directory. This cache data will be expired at 60 seconds after.

$ cat cache.d/items/1
  <ul>
    <li>AAA</li>
    <li>BBB</li>
    <li>CCC</li>
  </ul>

Logging

If you set logging object to tenjin.logger, Tenjin will report loading template files.

For example:

ex-logger.py
import tenjin
from tenjin.helpers import *

## set logging object
import logging
logging.basicConfig(level=logging.INFO)
tenjin.logger = logging

engine = tenjin.Engine()
context = {'name': 'World'}
html = engine.render('example.pyhtml', context)

If you run it first time, Tenjin will report that template object is stored into cache file.

$ python ex-logger.py > /dev/null
INFO:root:[tenjin.MarshalCacheStorage] store cache (file='/home/user/example.pyhtml.cache')

And if you run it again, Tenjin will report that template object is loaded from cache file.

$ python ex-logger.py > /dev/null
INFO:root:[tenjin.MarshalCacheStorage] load cache (file='/home/user/example.pyhtml.cache')

Using Tenjin with Google App Engine

Tenjin supports Google App Engine. All you have to do is just call tenjin.gae.init().

import tenjin
from tenjin.helpers import *
import tenjin.gae; tenjin.gae.init()

## it is recommended to configure logging
import logging
logging.basicConfig(level=logging.DEBUG)
tenjin.logger = logging

tenjin.gae.init() do the followings internally.

## change tenjin.Engine to cache template objects into memcache service
## (using CURRENT_VERSION_ID as namespace).
ver = os.environ.get('CURRENT_VERSION_ID').split('.')[0]
Engine.cache = tenjin.gae.GaeMemcacheCacheStorage(namespace=ver)
## change fragment cache store to use memcache service
fragcache = tenjin.helpers.fragment_cache
fragcache.store    = tenjin.gae.aGaeMemcacheStore(namespace=ver)
fragcache.lifetime = 60    #  1 minute
fragcache.prefix   = 'fragment.'

NOTICE: Google App Engine shares memcache in any version. In other words, Google App Engine doesn't allow to separate memcache for each version. This means that memcache data can be conflict between old and new version application in Google App Engine. Tenjin avoids this confliction by using version id as namespace.


Specify Function Names of escape() and to_str()

It is able to specify function names of escape() and to_str() which are used in converted Python script.

main.y
import tenjin
from tenjin.helpers import *
engine = tenjin.Engine(path=['views'], escapefunc='cgi.escape', tostrfunc='str')
print(engine.get_template('page.pyhtml').script)
views/page.pyhtml
<p>
  escaped:     ${value}
  not escaped: #{value}
</p>
Result
$ python main.py
_buf.extend(('''<p>
  escaped:     ''', cgi.escape(str(value)), '''
  not escaped: ''', str(value), '''
</p>\n''', ));

If you need faster version of to_str() and escape(), see this section.


Preprocessing

Tenjin supports preprocessing of template. Preprocessing executes some logics when templates are loaded and that logics are not executed when rendering. Preprocessing makes your application much faster.

Basics of Preprocessing

Notation of preprocessing:

<?PY ... ?>
Preprocessing statement.
#{{...}}
Preprocessing expression (without HTML escape)
${{...}}
Preprocessing expression (with HTML escape)

The following shows difference between ${ ... } and ${{ ... }}.

views/pp-example1.pyhtml
## normal expression
value = ${value}
## with preprocessing
value = ${{value}}
pp-example1.py
value = 'My Great Example'

## create engine object with preprocessing enabled
import tenjin
from tenjin.helpers import *
engine = tenjin.Engine(path=['views'], preprocess=True)

## print Python script code
print("------ converted script ------")
print(engine.get_template('pp-example1.pyhtml').script)

## render html
html = engine.render('pp-example1.pyhtml', {})
print("------ rendered html ------")
print(html)
Result: notice that ${{...}} is evaluated at template converting stage.
$ python pp-example1.py
------ converted script ------
_buf.extend(('''## normal expression
value = ''', escape(to_str(value)), '''
## with preprocessing
value = My Great Example\n''', ));

------ rendered html ------
## normal expression
value = My Great Example
## with preprocessing
value = My Great Example

You can confirm preprocessed template by 'pytenjin -P' command.

Preprocessed template
$ pytenjin -P -c 'value="My Great Example"' views/pp-example1.pyhtml
## normal expression
value = ${value}
## with preprocessing
value = My Great Example

If you want to see preprocessing script (not preprocessed script), use 'pytenjin -sP' command.

$ pytenjin -sP -c 'value="My Great Example"' views/pp-example1.pyhtml
_buf = []; _buf.extend(('''## normal expression
value = ${value}
## with preprocessing
value = ''', escape(to_str(_decode_params(value))), '''\n''', ));
print(''.join(_buf))

Loop Expantion

It is possible to evaluate some logics by '<?PY ?>' when convert template into Python script code. For example, you can expand loop in advance to improve performance.

views/pp-example2.pyhtml
<?PY states = { "CA": "California", ?>
<?PY            "NY": "New York", ?>
<?PY            "FL": "Florida",  ?>
<?PY            "TX": "Texas",  ?>
<?PY            "HI": "Hawaii", } ?>
<?PY # ?>
<?py chk = { params['state']: ' selected="selected"' } ?>
<?PY codes = states.keys() ?>
<?PY codes.sort() ?>
<select name="state">
  <option value="">-</option>
<?PY for code in codes: ?>
  <option value="#{{code}}"#{chk.get('#{{code}}', '')}>${{states[code]}}</option>
<?PY #endfor ?>
</select>

Preprocessed script code shows that loop is expanded in advance. It means that loop is not executed when rendering template.

Preprocessed template
$ pytenjin -P views/pp-example2.pyhtml
<?py chk = { params['state']: ' selected="selected"' } ?>
<select name="state">
  <option value="">-</option>
  <option value="CA"#{chk.get('CA', '')}>California</option>
  <option value="FL"#{chk.get('FL', '')}>Florida</option>
  <option value="HI"#{chk.get('HI', '')}>Hawaii</option>
  <option value="NY"#{chk.get('NY', '')}>New York</option>
  <option value="TX"#{chk.get('TX', '')}>Texas</option>
</select>

Parameters

Assume that link_to() is a helper method which takes label and url and generate <a></a> tag. In this case, label and url can be parameterized by _p("...") and _P("..."). The former is converted into #{...} and the latter converted into ${...} by preprocessor.

views/pp-example3.pyhtml
<?PY ## ex. link_to('Show', '/show/1')  => <a href="/show/1">Show</a> ?>
<?PY def link_to(label, url): ?>
<?PY     import urllib ?>
<?PY     return '<a href="%s">%s</a>' % (urllib.quote(url), label) ?>
<?PY # ?>
#{{link_to('Show '+_P('params["name"]'), '/items/show/'+_p('params["id"]'))}}

The following shows that _P('...') and _p('...') are converted into ${...} and #{...} respectively.

Preprocessed template:
$ pytenjin -P views/pp-example3.pyhtml
<a href="/items/show/#{params["id"]}">Show ${params["name"]}</a>

There are many web-application framework and they provides helper functions. These helper functions are divided into two groups. link_to() or _() (function for M17N) return the same result when the same arguments are passed. These functions can be expanded by preprocessor. Some functions return the different result even if the same arguments are passed. These functions can't be expaned by preprocessor.

Preprocessor has the power to make view-layer much faster, but it may make the debugging difficult. You should use it carefully.




pytenjin Command

See 'pytenjin -h' for details.

Syntax Check

Command-line option '-z' checks syntax of template files.

example.pyhtml
<ul>
<?py for item in items: ?>
 <li>${item}</li>
<?py   #endfor ?>
</ul>
Result:
$ pytenjin -z example.pyhtml
example.pyhtml:4:3: unindent does not match any outer indentation level
  4:   #endfor
       ^

Error message is the same format as gcc compiler or java compiler. Error jump in Emacs or other editor is available.

Command-line option '-q' (quiet-mode) prints nothing if it has no errors.


Convert Template into Python Script

Command-line option '-s' converts template file into Python script code.

example.pyhtml
<ul>
<?py for item in items: ?>
  <li>${item}</li>
<?py #endfor ?>
</ul>
Result (-s)
$ pytenjin -s example.pyhtml
_buf = []; _buf.extend(('''<ul>\n''', ));
for item in items:
    _buf.extend(('''  <li>''', escape(to_str(item)), '''</li>\n''', ));
#endfor
_buf.extend(('''</ul>\n''', ));
print(''.join(_buf))

Option '-b' removes preamble ('_buf = []') and postamble ('print "".join(_buf)').

Result (-sb)
$ pytenjin -sb example.pyhtml
_buf.extend(('''<ul>\n''', ));
for item in items:
    _buf.extend(('''  <li>''', escape(to_str(item)), '''</li>\n''', ));
#endfor
_buf.extend(('''</ul>\n''', ));

Retrieve Embedded Code

Tenjin allows you to retrieve embedded code from template files. This is aimed to help debugging template.

It is hard to debug large template files because HTML and embedded code are mixed in a file. Retrieving embedded code from template files will help you to debug large template files.

Assume the following template file.

example.pyhtml
<table>
<?py i = 0 ?>
<?py for item in items: ?>
<?py     i += 1 ?>
  <tr>
    <td>#{i}</td>
    <td>${item}</td>
  </tr>
<?py #endfor ?>
</table>

Option '-S' (or '-a retrieve') retrieves embedded codes.

Result (-Sb)
$ pytenjin -Sb example.pyhtml

i = 0
for item in items:
    i += 1

        to_str(i); 
        escape(to_str(item)); 

#endfor

Option '-X' (or '-a statements') retrieves only statements.

Result (-Xb)
$ pytenjin -Xb example.pyhtml

i = 0
for item in items:
    i += 1




#endfor

Option '-N' adds line numbers.

Result (-NXb)
$ pytenjin -NXb example.pyhtml
    1:  
    2:  i = 0
    3:  for item in items:
    4:      i += 1
    5:  
    6:  
    7:  
    8:  
    9:  #endfor
   10:  

Option '-U' (unique) compress empty lines.

Result (-UNXb)
$ pytenjin -UNXb example.pyhtml

    2:  i = 0
    3:  for item in items:
    4:      i += 1

    9:  #endfor

Option '-C' (compact) removes empty lines.

Result (-CNXb)
$ pytenjin -CNXb example.pyhtml
    2:  i = 0
    3:  for item in items:
    4:      i += 1
    9:  #endfor

Execute Template File

You can execute template file in command-line.

example.pyhtml
<?py items = ['<AAA>', 'B&B', '"CCC"'] ?>
<ul>
<?py for item in items: ?>
  <li>${item}</li>
<?py #endfor ?>
</ul>
Result
$ pytenjin example.pyhtml
<ul>
  <li>&lt;AAA&gt;</li>
  <li>B&amp;B</li>
  <li>&quot;CCC&quot;</li>
</ul>

Context Data

You can specify context data with command-line option '-c'.

example.pyhtml
<ul>
<?py for item in items: ?>
  <li>${item}</li>
<?py #endfor ?>
</ul>
Result
$ pytenjin -c 'items=["A","B","C"]' example.pyhtml
<ul>
  <li>A</li>
  <li>B</li>
  <li>C</li>
</ul>

If you want to specify several values, separate them by ';' such as '-c "x=10; y=20"'.

If you installed PyYAML library, you can specify context data in YAML format. Tenjin regards context data string as YAML format if it starts with '{'.

Result
$ pytenjin -c '{items: [A, B, C]}' example.pyhtml
<ul>
  <li>A</li>
  <li>B</li>
  <li>C</li>
</ul>

In addition, Tenjin supports context data file in Python format or YAML format.

context.py
items = [
   "AAA",
   123,
   True,
]
Result
$ pytenjin -f context.py example.pyhtml
<ul>
  <li>AAA</li>
  <li>123</li>
  <li>True</li>
</ul>
context.yaml
items:
  - AAA
  - 123
  - true
Result
$ pytenjin -f context.yaml example.pyhtml
<ul>
  <li>AAA</li>
  <li>123</li>
  <li>True</li>
</ul>


Trouble shooting

I got an SyntaxError exception.

Command-line option '-z' checks syntax of template file. You should check template by it.

File 'syntaxerr.pyhtml':

<?py for i in range(0, 10): ?>
<?py     if i % 2 == 0: ?>
#{i} is even.
<?py     else ?>
#{i} is odd.
<?py     #end ?>
<?py #end ?>

Result:

$ pytenjin -z syntaxerr.pyhtml
syntaxerr.pyhtml:4:9: invalid syntax
  4:     else
             ^

I got 'SyntaxError: encoding declaration in Unicode string'

This is because you added encoding declaration in template file AND you specified encoding option to tenjin.Engine() or tenjin.Template().

Solution:


I got UnicodeDecodeError, but I can't find what is wrong

If you got UnicodeDecodeError, you should do the following solutions.



Tips

Flexible Indentation

You may want Tenjin to be more flexible about indentation of statements. For example, You may not like such as:

<html>
  <body>
    <div id="main-content">
<?py if items: ?>
      <table>
        <tbody>
    <?py i = 0 ?>
    <?py for item in items: ?>
    <?py     i += 1 ?>
          <tr>
            <td>${i}</td>
        <?py if item: ?>
            <td>${item}</td>
        <?py else: ?>
            <td>-</td>
        <?py #endif ?>
          </tr>
    <?py #endfor ?>
        </tbody>
      </table>
<?py #endif ?>
    </div>
  </body>
</html>

And you may prefer the following:

File 'flexibleindent.pyhtml':
<html>
  <body>
    <div id="main-content">
      <?py if items: ?>
      <table>
        <tbody>
          <?py i = 0 ?>
          <?py for item in items: ?>
          <?py   i += 1 ?>
          <tr>
            <td>${i}</td>
            <?py if item: ?>
            <td>${item}</td>
            <?py else: ?>
            <td>-</td>
            <?py #endif ?>
          </tr>
        <?py #endfor ?>
        </tbody>
      </table>
      <?py #endif ?>
    </div>
  </body>
</html>

If you want Tenjin to be more flexible about indentation, try http://gist.github.com/129297. Save this code as 'my_template.py' and try the following:

File 'flexibleindent.py':
import tenjin
from tenjin.helpers import *
from my_template import MyTemplate
tenjin.Engine.templateclass = MyTemplate

import sys
template_name = len(sys.argv) > 1 and sys.argv[1] or 'flexibleindent.pyhtml'
engine = tenjin.Engine()
print("-------------------- script")
print(engine.get_template(template_name).script)
print("-------------------- html")
html = engine.render(template_name, {'items': ['AAA', None, 'CCC']})
print(html)
Result:
$ python flexibleindent.py flexibleindent.pyhtml
-------------------- script
_buf.extend(('''<html>
  <body>
    <div id="main-content">\n''', ));
if items:
    _buf.extend(('''      <table>
        <tbody>\n''', ));
    i = 0
    for item in items:
        i += 1
        _buf.extend(('''          <tr>
            <td>''', escape(to_str(i)), '''</td>\n''', ));
        if item:
            _buf.extend(('''            <td>''', escape(to_str(item)), '''</td>\n''', ));
        else:
            _buf.extend(('''            <td>-</td>\n''', ));
        #endif
        _buf.extend(('''          </tr>\n''', ));
    #endfor
    _buf.extend(('''        </tbody>
      </table>\n''', ));
#endif
_buf.extend(('''    </div>
  </body>
</html>\n''', ));

-------------------- html
<html>
  <body>
    <div id="main-content">
      <table>
        <tbody>
          <tr>
            <td>1</td>
            <td>AAA</td>
          </tr>
          <tr>
            <td>2</td>
            <td>-</td>
          </tr>
          <tr>
            <td>3</td>
            <td>CCC</td>
          </tr>
        </tbody>
      </table>
    </div>
  </body>
</html>

NOTE: MyTemplate will be the default template class in the next release (1.0.0).


Webext

I have to say that bottleneck of Tenjin is calling to_str() and escape(), not string concatenation. Therefore if you want to make Tenjin much faster, you must make to_str() and escape() faster (or omit calling them).

For example, using str() instead of to_str() will make Tenjin much faster. The following benchmark result shows that str()(= 'tenjin-str') is much faster than to_str()(= 'tenjin').

str() is faster than to_str()
$ cd Tenjin-X.X.X/benchmark
$ python -V
Python 2.5.5
$ python bench.py -q -n 10000 tenjin tenjin-str
*** ntimes=10000
                                    utime     stime     total      real
tenjin                             3.7500    0.0400    3.7900    3.7936
tenjin-str                         2.4500    0.0300    2.4800    2.4857

But str() doesn't return empty string if argument is None. In addition str() raises UnicodeEncodeError frequently.

Other solution is Webext. Webext is an extension module which implement to_str() and escape() in C language.

import tenjin
from tenjin.helpers import *
from webext import to_str, escape    # use webext's functions instead of tenjin's

Benchmark script in Tenjin already supports Webext. It shows that Webext makes Tenjin much faster especially html escaping.

Intel CoreDuo2 2GHz, Mac OS X 10.6, Python 2.5.5
### without html escaping
$ cd Tenjin-X.X.X/
$ cd benchmark/
$ python bench.py -q -n 10000 tenjin tenjin-str tenjin-webext
*** ntimes=10000
                                    utime     stime     total      real
tenjin                             3.8100    0.0400    3.8500    3.8462
tenjin-str                         2.4500    0.0200    2.4700    2.4815
tenjin-webext                      2.4500    0.0300    2.4800    2.4825

## with html escaping
$ python bench.py -e -q -n 10000 tenjin tenjin-str tenjin-webext
*** ntimes=10000
                                    utime     stime     total      real
tenjin                             7.2900    0.0500    7.3400    7.4669
tenjin-str                         5.7400    0.0400    5.7800    5.8202
tenjin-webext                      2.9700    0.0400    3.0100    3.0079

M17N Page

If you have M17N site, you can make your site faster by Tenjin.

In M17N-ed site, message translation function (such as _('message-key')) is called many times. Therefore if you can eliminate calling that function, your site can be faster. And Tenjin can do it by preprocessing.

The point is:

The following is an example to generate M17N pages from a template file.

m18n.pyhtml:
<div>
<?PY ## '_()' represents translator method ?>
 <p>${{_('Hello')}} ${username}!</p>
</div>
m18n.py:
# -*- coding: utf-8 -*-
import tenjin
from tenjin.helpers import *
import re

##
## message catalog to translate message
##
MESSAGE_CATALOG = {
    'en': { 'Hello': 'Hello',
            'Good bye': 'Good bye',
          },
    'fr': { 'Hello': 'Bonjour',
            'Good bye': 'Au revoir',
          },
}

##
## create translation function and return it.
## ex.
##    _ = create_m18n_func('fr')
##    print _('Hello')   #=> 'Bonjour'
##
def create_m18n_func(lang):
    dct = MESSAGE_CATALOG.get(lang)
    if not dct:
        raise ValueError("%s: unknown lang." % lang)
    def _(message_key):
        return dct.get(message_key)
    return _
    # or return dct.get
    
##
## cache storage class to cache template object for each language
##
class M17nCacheStorage(tenjin.MarshalCacheStorage):

    lang = 'en'       # default language

    def __init__(self, *args, **kwargs):
        if 'lang' in kwargs:
            lang = kwargs.pop('lang')
            if lang: 
                self.lang = lang
        tenjin.MarshalCacheStorage.__init__(self, *args, **kwargs)

    ## change cache filename to 'file.pyhtml.lang.cache'
    def _cachename(self, fullpath):
        return "%s.%s.cache" % (fullpath, self.lang)

##
## test program
##
if __name__ == '__main__':

    ## create cache storage and engine for English
    m17ncache = M17nCacheStorage(lang='en')
    engine_en = tenjin.Engine(preprocess=True, cache=m17ncache)

    ## render html for English
    context = { 'username': 'World' }
    context['_'] = create_m18n_func('en')
    html = engine_en.render('m18n.pyhtml', context)
    print("--- lang: en ---")
    print(html)
    
    ## create cache storage and engine for French
    m17ncache = M17nCacheStorage(lang='fr')
    engine_fr = tenjin.Engine(preprocess=True, cache=m17ncache)

    ## render html for French
    context = { 'username': 'World' }
    context['_'] = create_m18n_func('fr')
    html = engine_fr.render('m18n.pyhtml', context)
    print("--- lang: fr ---")
    print(html)
Result:
$ python m18n.py
--- lang: en ---
<div>
 <p>Hello World!</p>
</div>

--- lang: fr ---
<div>
 <p>Bonjour World!</p>
</div>

After that, you can find two cache files are created.

$ ls m18n.pyhtml*
m18n.pyhtml    m18n.pyhtml.en.cache    m18n.pyhtml.fr.cache

And each cache files have different content.

### "_('Hello')" is translated into "Hello" in Engilish cache file
$ pytenjin -a dump m18n.pyhtml.en.cache
_buf.extend(('''<div>
 <p>Hello ''', escape(to_str(username)), '''!</p>
</div>\n''', ));

### "_('Hello')" is translated into "Bonjour" in French cache file
$ pytenjin -a dump m18n.pyhtml.fr.cache
_buf.extend(('''<div>
 <p>Bonjour ''', escape(to_str(username)), '''!</p>
</div>\n''', ));

Template Inheritance

Tenjin doesn't support Template Inheritance which Django template engine does. But you can emulate it by capturing(*2). See this section for details.

(*2)
Notice that capturing is useful but not so powerful than template inheritance.