pyTenjin User's Guide
last update: $Date$
Table of Contents:
- Overview
- Basic Examples
- Advanced Features
-
pytenjin
Command - Trouble shooting
- Tips
Overview
Tenjin is a very fast and full-featured template engine implemented in pure Python.
ATTENTION: pyTenjin will change some featrues in the next release. See Planned Changes in the Next Relase section for details.
Features
-
Very fast
- About 10 times faster than Django template
- About 4 times faster than Cheetah
- About 2 times faster than Mako
- Full featured
- Easy to learn
- Because you can embed any Python statements or expression in your template files
- You don't have to study template-specific language
- Compact
- Only 1500 lines of code
- Very ligthweight to load module (important for CGI script and Google App Engine)
- Supports Google App Engine
Install
pyTenjin supports Python 2.3 or later. Python 3 is also supported.
You can install Tenjin by easy_install command.
$ sudo easy_install Tenjin
Or download Tenjin-X.X.X.tar.gz, extract it, and type 'python setup.py install'.
$ wget http://downloads.sourceforge.net/project/tenjin/pytenjin/X.X.X/Tenjin-X.X.X.tar.gz $ tar xzf Tenjin-X.X.X.tar.gz $ cd Tenjin-X.X.X/ $ sudo python setup.py install
Benchmark
Tenjin package contains benchmark program.
$ cd pytenjin-X.X.X/benchmark $ python -V Python 2.5.5 $ python bench.py -q -n 10000 *** ntimes=10000 compiling bench_cheetah.tmpl ... Compiling bench_cheetah.tmpl -> bench_cheetah.py (backup bench_cheetah.py.bak) *** loading context data (file=bench_context.py)... *** start benchmark *** ntimes=10000 utime stime total real tenjin 3.8700 0.0000 3.8700 3.8858 tenjin-create 4.9600 0.5600 5.5200 5.5457 tenjin-str 2.4900 0.0000 2.4900 2.4967 tenjin-webext 2.3500 0.0000 2.3500 2.3703 django 70.5100 0.0500 70.5600 70.8199 django-create 83.7900 0.3500 84.1400 84.5050 cheetah 15.6900 0.0200 15.7100 15.7504 cheetah-create 16.6700 0.0100 16.6800 16.7545 kid 287.4500 0.2900 287.7400 289.1525 kid-create 288.6500 0.4800 289.1300 290.7973 genshi 208.2000 0.2400 208.4400 209.4677 genshi-create 452.4900 1.5200 454.0100 456.4957 mako 7.0500 0.0400 7.0900 7.1348 mako-create 9.7300 0.8100 10.5400 10.5842 mako-nocache 110.9300 0.6200 111.5500 112.1807 templetor 47.5300 0.0700 47.6000 47.8334 templetor-create 362.8200 2.0100 364.8300 367.4523 jinja2 7.6800 0.0600 7.7400 8.1972 jinja2-create 120.5900 0.6200 121.2100 122.0719
Versions:
- Python 2.5.5
- Tenjin 0.9.0
- Django 1.1.0
- Cheetah 2.2.2
- Kid 0.9.6
- Genshi 0.5.1
- Mako 0.2.5
- Templetor (web.py) 0.32
- Jinja2 2.2.1
This shows the followings.
- Tenjin is the fastest template engine.
- Cheetah's performance is good.
- Django's performance is not good.
- Kid's performance is worse. It is too slow.
- Genshi's performance is also worse. It is faster than Kid, but slower than others.
- Mako's performance is very good when module caching is enabled.
- Templetor's performance is not good for mod_python and worse for CGI program.
- Jinja's performance is very good if you can cache template object, but sad performance for CGI program.
Basic Examples
This section describes basics of Tenjin.
Render Template
Notation:
- <?py ... ?>
- Python statements
- ${...}
- Python expression (with HTML escape)
- #{...}
- Python expression (without HTML escape)
<h2>${title}</h2> <table> <?py i = 0 ?> <?py for item in items: ?> <?py i += 1 ?> <?py klass = i % 2 and 'odd' or 'even' ?> <tr class="#{klass}"> <td>${item}</td> </tr> <?py #endfor ?> </table>
## import modules and helper functions import tenjin from tenjin.helpers import * ## context data context = { 'title': 'Tenjin Example', 'items': ['<AAA>', 'B&B', '"CCC"'], } ## cleate engine object engine = tenjin.Engine(path=['views']) ## render template with context data html = engine.render('page.pyhtml', context) print(html)
$ python main.py <h2>Tenjin Example</h2> <table> <tr class="odd"> <td><AAA></td> </tr> <tr class="even"> <td>B&B</td> </tr> <tr class="odd"> <td>"CCC"</td> </tr> </table>
Tips: you can check template syntax by 'pytenjin -z'.
$ pytenjin -z views/*.pyhtml views/page.pyhtml - ok.
Template Syntax
You can write any Python statements or expression in yor template file, but there are several restrictions.
- It is strongly recommended to close 'for', 'if', 'while', ... by corresponding '#endfor', '#endif', '#endwhile', and so on. This restriction is not enabled in current release, but it will be enabled in the next release (1.0).
- Statement should be indented with 4 spaces. (This restriction will be removed in the next release. See Flexible Indentation section for details.)
- Conditional expression of 'if', 'while', ... should be in a line.
- '#' (comment) after ':' or '#endXXX' are not allowed.
## NG because 'for' is not closed by '#endfor' <ul> <?py for item in items: ?> <li>${item}</li> <?py #end ?> </ul> ## NG because indent is 2 spaces <?py if x > 0: ?> <?py if y > 0: ?> <?py if z > 0: ?> <?py #endif ?> <?py #endif ?> <?py #endif ?> ## NG because conditional expression of 'if' is not in a line <?py if re.search(r'^\[(\w+)\]',?> <?py line, re.M): ?> <p>matched.</p> <?py #endif ?> ## NG because there is a comment after ':' and '#endfor' <ul> <?py if item in items: ## beginning of loop ?> <li>${item}</li> <?py #endfor ## end of loop ?> </ul>
You may feel that indent restriction of Tenjin is strong and not convenient. If you feel so, see Flexible Indentation section.
Show Converted Source Code
pyTenjin converts template files into Python script and executes it. Compiled Python script is saved into cache file automatically. Cache file is saved with Python's marshal format (= bytecode format).
$ ls views/page.pyhtml* views/page.pyhtml views/page.pyhtml.cache $ file views/page.pyhtml.cache views/page.pyhtml.cache: data
You can get converted script by 'pytenjin -s
'.
$ pytenjin -s views/page.pyhtml _buf = []; _buf.extend(('''<h2>''', escape(to_str(title)), '''</h2> <table>\n''', )); i = 0 for item in items: i += 1 klass = i % 2 and 'odd' or 'even' _buf.extend((''' <tr class="''', to_str(klass), '''"> <td>''', escape(to_str(item)), '''</td> </tr>\n''', )); #endfor _buf.extend(('''</table>\n''', )); print(''.join(_buf))
If you specify '-sb
' instead of '-s
', neither preamble (= '_buf = [];
') nor postamble (= 'print "".join(_buf)
') are printed. See Retrieve Embedded Code section for more information.
Or:
import tenjin template = tenjin.Template('views/page.pyhtml') print(template.script) ### or: ## engine = tenjin.Engine() ## print(engine.get_template('page.pyhtml').script)
Layout Template
Layout template will help you to use common HTML design for all pages.
<!DOCTYPE> <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> <title>${title}</title> </head> <body> #{_content} </body> </html>
## import modules and helper functions import tenjin from tenjin.helpers import * ## context data context = { 'title': 'Tenjin Example', 'items': ['<AAA>', 'B&B', '"CCC"'], } ## cleate engine object engine = tenjin.Engine(path=['views'], layout='_layout.pyhtml') ## render template with context data html = engine.render('page.pyhtml', context) print(html)
$ python main.py <!DOCTYPE> <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> <title>Tenjin Example</title> </head> <body> <h2>Tenjin Example</h2> <table> <tr class="odd"> <td><AAA></td> </tr> <tr class="even"> <td>B&B</td> </tr> <tr class="odd"> <td>"CCC"</td> </tr> </table> </body> </html>
You can specify other layout template file with render()
method.
## use other layout template file engine.render('page.pyhtml', context, layout='_other_layout.pyhtml') ## don't use layout template file engine.render('page.pyhtml', context, layout=False)
Tenjin supports nested layout template. See Nested Layout Template section for details.
Context Variables
'_context
' variable contains context dictionary which is passed from main program to template file.
Using '_context
' dictionary, you can pass any data from template file to main program and layout template file.
<!DOCTYPE> <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> <title>${page_title}</title> </head> <body> #{_content} </body> </html>
<?py _context['page_title'] = 'Tenjin: Layout Template Example' ?> <h2>${title}</h2> <table> <?py i = 0 ?> <?py for item in items: ?> <?py i += 1 ?> <?py klass = i % 2 and 'odd' or 'even' ?> <tr class="#{klass}"> <td>${item}</td> </tr> <?py #endfor ?> </table>
$ python main.py <!DOCTYPE> <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> <title>Tenjin: Layout Template Example</title> </head> <body> <h2>Tenjin Example</h2> <table> <tr class="odd"> <td><AAA></td> </tr> <tr class="even"> <td>B&B</td> </tr> <tr class="odd"> <td>"CCC"</td> </tr> </table> </body> </html>
Template Arguments
It is recommended to declare context variables in your template files for readability reason.
<?py #@ARGS title, items ?> <?py _context['page_title'] = 'Tenjin: Layout Template Example' ?> <h2>${title}</h2> <table> <?py i = 0 ?> <?py for item in items: ?> <?py i += 1 ?> <?py klass = i % 2 and 'odd' or 'even' ?> <tr class="#{klass}"> <td>${item}</td> </tr> <?py #endfor ?> </table>
$ pytenjin -s views/page.pyhtml _buf = [] title = _context.get('title'); items = _context.get('items'); _context['page_title'] = 'Tenjin: Layout Template Example' _buf.extend(('''<h2>''', escape(to_str(title)), '''</h2> <table>\n''', )); i = 0 for item in items: i += 1 klass = i % 2 and 'odd' or 'even' _buf.extend((''' <tr class="''', to_str(klass), '''"> <td>''', escape(to_str(item)), '''</td> </tr>\n''', )); #endfor _buf.extend(('''</table>\n''', )); print(''.join(_buf))
<?py #@ARGS _content, page_title ?> <!DOCTYPE> <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> <title>${page_title}</title> </head> <body> #{_content} </body> </html>
$ pytenjin -s views/_layout.pyhtml _buf = [] _content = _context.get('_content'); page_title = _context.get('page_title'); _buf.extend(('''<!DOCTYPE> <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> <title>''', escape(to_str(page_title)), '''</title> </head> <body> ''', to_str(_content), ''' </body> </html>\n''', )); print(''.join(_buf))
Tips: If you want to specify default value of context variable, try _context.get('varname', defaultvalue)
.
<?py ## if username is not specified, use 'World' as default. username = _context.get('username', 'World') ?> <p>Hello ${username}</p>
Include Partial Template
You can include other template files by include()
helper function.
- include(template-name, **kwargs)
- Include other template. template-name can be file name or template short name. kwargs is passed to template as local variables.
<?py #@ARGS _content, page_title ?> <!DOCTYPE> <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> <title>${page_title}</title> </head> <body> <?py include('_header.pyhtml', title=page_title) ?> #{_content} <?py include('_footer.pyhtml') ?> </body> </html>
<?py #@ARGS title ?> <div class="header"> <h1>${title}</h1> </div>
<?py #@ARGS ?> <address> copyright(c) 2010 kuwata-lab.com all rights reserved </address>
$ python main.py <!DOCTYPE> <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> <title>Tenjin: Layout Template Example</title> </head> <body> <div class="header"> <h1>Tenjin: Layout Template Example</h1> </div> <h2>Tenjin Example</h2> <table> <tr class="odd"> <td><AAA></td> </tr> <tr class="even"> <td>B&B</td> </tr> <tr class="odd"> <td>"CCC"</td> </tr> </table> <address> copyright(c) 2010 kuwata-lab.com all rights reserved </address> </body> </html>
Template Short Name
If you specify template postfix, you can use template short name such as ':views/page
' instead of 'views/page.pyhtml
'. Notice that template short name should start with ':
'.
## import modules and helper functions import tenjin from tenjin.helpers import * ## context data context = { 'title': 'Tenjin Example', 'items': ['<AAA>', 'B&B', '"CCC"'], } ## cleate engine object engine = tenjin.Engine(path=['views'], postfix='.pyhtml', layout=':_layout') ## render template with context data html = engine.render(':page', context) print(html)
<?py #@ARGS _content, page_title ?> <!DOCTYPE> <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> <title>${page_title}</title> </head> <body> <?py include(':_header') ?> #{_content} <?py include(':_footer') ?> </body> </html>
Template Encoding
If you want to specify encoding name of template file, add magic comment at 1st line in template file.
<?py # -*- coding: utf-8 -*- ?> <html> ...
Python 2.x
Tenjin provides two approaches for encoding in Python 2.x.
- (A) Binary-based approach
-
Tenjin handles templates as binary file. For example, text "foo" will be converted into
_buf.exnted(('foo', ))
. In this approach, unicode object should be encoded into binary(=str) byto_str()
.to_str = tenjin.generate_tostrfunc(encode='utf-8') # ex. to_str(u'\u65e5\u672c\u8a9e') => '\xe6\x97\xa5\xe6\x9c\xac\xe8\xaa\x9e' engine = tenjin.Engine()
If you have some troubles about encoding in this approach, add magic comment into your template file.
<?py # -*- coding: utf-8 -*- ?> <h1>Hello</h1>
- (B) Unicode-based approach
-
You can hanlde templates as unicode string with
encoding
option fortenjin.Engine()
. For example, text "foo" will be converted into_buf.extend((u'foo', ))
. In this approach, binary(=str) should be decoded into unicode into_str()
.to_str = tenjin.generate_tostrfunc(decode='utf-8') # ex. to_str('\xe6\x97\xa5\xe6\x9c\xac\xe8\xaa\x9e') #=> u'\u65e5\u672c\u8a9e' engine = tenjin.Engine(encoding='utf-8') html = engine.render('index.pyhtml') if isinstance(html, unicode): html = html.encode('utf-8') print(html)
You should not add encoding declaration in your templates, or you will get the following error.
SyntaxError: encoding declaration in Unicode string
In Python 2.x, binary-based approach is faster than unicode-based approach. If you don't have any special reason, it is recommend to be with binary-based approach.
Python 3.0
In Python 3.0, string is treated as unicode object. So Tenjin handles all templates in string base (= unicode base), and bytes data is converted into str by to_str() function. If you don't specify any encoding, Tenjin uses 'utf-8' encoding as default.
to_str = tenjin.generate_tostrfunc('utf-8') ## decode bytes into unicode engine = tenjin.Engine(encoding='utf-8') ## handle template as unicode
Helper Function
Tenjin provides some modules for helper functions.
tenjin.helpers module
Module tenjin.helpers
provides basic helper functions.
- to_str(value)
-
Converts value into string. None is converted into empty string instead of "None". Unicode object is encoded into string with 'utf-8' encoding name. If you want to use other encoding name, use generate_tostr_func().
>>> from tenjin.helpers import to_str >>> to_str(123) '123' >>> to_str(None) # None is converted into empty string '' >>> to_str(u'日本語') # unicode object is encoded into str '\xc3\xa6\xc2\x97\xc2\xa5\xc3\xa6\xc2\x9c\xc2\xac\xc3\xa8\xc2\xaa\xc2\x9e'
- escape(str)
-
Escape HTML special characters.
>>> from tenjin.helpers import escape >>> escape('< > & "') '< > & "'
- generate_tostrfunc(encode=encoding, decode=encoding)
-
Generate to_str() function with enoding name.
>>> from tenjin.helpers import generate_tostrfunc >>> ## generate to_str() function which encodes unicode into binary(=str). >>> to_str = generate_tostrfunc(encode='utf-8') >>> to_str(u'hoge') 'hoge' >>> ## generate to_str() function which decodes binary(=str) into unicode. >>> to_str = generate_tostrfunc(decode='utf-8') >>> to_str('hoge') u'hoge'
- echo(value)
-
Add value string into _buf. This is similar to echo() function of PHP.
## add _content into here <?py echo(_content) ?> ## this is same as #{...} #{_content}
- not_cached(key, lifetime=0)
- If fragment is expired or not cached, start caching fragment with key and lifetime (seconds), and returns True. If fragment is already cached with key, returns False. See Fragment Cache for details.
- echo_cached()
- Echo cached fragment. If caching is started by not_cached, stop and cache it. This function should be used with not_cached(). See Fragment Cache for details.
- start_capture(name), stop_capture()
- Start capturing with specified name. See Capturing section for details.
- captured_as(name)
- Return True if captured with name. See Capturing section for details.
- _P(value), _p(value)
- Helper method for preprocessing. See Preprocessing section for details.
tenjin.helpers.html module
Module tenjin.helpers.html
provides HTML specific helper functions.
- escape(str), escape_xml(str)
- Escapes HTML special characters. Same as
tejin.helpers.escape()
.
- checked(value)
-
Returns
' checked="checked"'
if value is true value, else returns empty string.>>> from tenjin.helpers.html import * >>> checked(1+1==2) ' checked="checked"' >>> checked(1+1==3) ''
- selected(value)
-
Returns
' selected="selected"'
if value is true value, else returns empty string.>>> from tenjin.helpers.html import * >>> selected(1+1==2) ' selected="selected"' >>> selected(1+1==3) ''
- disabled(value)
-
Returns
' disabled="disabled"'
if value is true value, else returns empty string.>>> from tenjin.helpers.html import * >>> disabled(1+1==2) ' disabled="disabled"' >>> disabled(1+1==3) ''
- new_cycle(*values)
-
Generates cycle object.
>>> from tenjin.helpers.html import * >>> cycle = new_cycle('odd', 'even') >>> cycle() 'odd' >>> cycle() 'even' >>> cycle() 'odd' >>> cycle() 'even'
- nl2br(str)
-
Replaces
"\n"
into"<br />\n"
.>>> from tenjin.helpers.html import * >>> nl2br("foo\nbar\nbaz\n") 'foo<br />\nbar<br />\nbaz<br />\n'
- text2html(str)
-
(experimental) Escapes xml characters and replace
"\n"
into"<br />\n"
.>>> from tenjin.helpers.html import * >>> text2html('<AAA>\nB&B\n"CCC"\n') '<AAA><br />\nB&B<br />\n"CCC"<br />\n'
- tagattr(name, expr, value=None, escape=True)
-
(experimental) Returns
' name="value"'
if expr is true value, else''
(empty string). If value is not specified, expr is used as value instead.>>> from tenjin.helpers.html import * >>> tagattr('name', 'account') ' name="account"' >>> tagattr('name', None) '' >>> tagattr('checked', True, 'checked') ' checked="checked"'
- tagattrs(**kwargs)
-
(experimental) Builds html tag attribtes.
>>> from tenjin.helpers.html import * >>> tagattrs(klass='main', size=20) ' class="main" size="20"' >>> tagattrs(klass='', size=0) ''
- nv(name, value, sep=None, **kwargs)
-
(experimental) Builds name and value attributes.
>>> from tenjin.helpers.html import * >>> nv('rank', 'A') 'name="rank" value="A"' >>> nv('rank', 'A', '-') 'name="rank" value="A" id="rank-A"' >>> nv('rank', 'A', '-', checked=True) 'name="rank" value="A" id="rank-A" checked="checked"' >>> nv('rank', 'A', '-', klass='error', style='color:red') 'name="rank" value="A" id="rank-A" class="error" style="color:red"'
Planned Changes in the Next Release (1.0.0)
This section describes planned changes in the next release (1.0.0).
- Indentation will be flexible. In other words, http://gist.github.com/129297 will be the default template class. See this section for details.
- 'if', 'for', 'while', ... should be closed by corresponding '#endif', '#endfor', '#endwhile', and so on. This restriction is necessary to allow flexible indentation.
- tenjin.GaeMemcacheCacheStorage will be moved to tenjin.gae module. Currently both tenjin.GaeMemcacheCacheStorage and tenjin.gae.GaeMemcacheCacheStorage are available. In the next release the former will be not available. Therefore it is recommended to use the latter instead the former.
- (Maybe) '_buf.extend(...)' will be changed to '_extend = _buf.extend; _extend(...)'. This change will make Tenjin a little faster (about 2 or 3 %). But this may not be introduced because performance gain is very small(*1).
- (*1)
- In Tenjin, string concatenation is not a bottleneck.
Advanced Features
Nested Layout Template
It is able to nest several layout template files.
<?py #@ARGS _content ?> <html> <body> #{_content} </body> </html>
<?py #@ARGS _content, title ?> <?py _context['_layout'] = '_site_layout.pyhtml' ?> <h2>${title}</h2> <!-- content --> #{_content} <!-- /content -->
<?py #@ARGS post_content ?> <?py _context['_layout'] = '_blog_layout.pyhtml' ?> <div class="article"> #{text2html(post_content)} </div>
import tenjin from tenjin.helpers import * from tenjin.helpers.html import text2html engine = tenjin.Engine(path=['views']) context = { 'title': 'Blog Post Test', 'post_content': "Foo\nBar\nBaz", } html = engine.render('blog_post.pyhtml', context) print(html)
$ python main.py <html> <body> <h2>Blog Post Test</h2> <!-- content --> <div class="article"> Foo<br /> Bar<br /> Baz </div> <!-- /content --> </body> </html>
Capturing
It is able to capture parital of template. It can be poor-man's alternative of Django's template-inheritance.
<?py #@ARGS blog_post, recent_posts ?> <h2>#{blog_post['title']}</h2> <div class="blog-post"> #{text2html(blog_post['content'])} </div> <?py start_capture('sidebar') ?> <h3>Recent Posts</h3> <ul> <?py for post in recent_posts: ?> <a href="/blog/#{post['id']}">${post['title']}</a> <?py #endfor ?> </ul> <?py stop_capture() ?>
<html> <body> <div id="header-part"> <?py if not captured_as('header'): ?> <h1>My Great Blog</h1> <?py #endif ?> </div> <div id="main-content"> #{_content} </div> <div id="sidebar-part"> <?py if not captured_as('sidebar'): ?> <h3>Links</h3> <ul> <a href="http://google.com/">Google</a> <a href="http://yahoo.com/">Yahoo!</a> </ul> <?py #endif ?> </div> </body> </html>
## context data blog_post = { 'title': 'Tenjin is Great', 'content': """ Tenjin has great features. - Very Fast - Full Featured - Easy to Use """[1:] } recent_posts = [ {'id': 1, 'title': 'Tenjin is Fast' }, {'id': 2, 'title': 'Tenjin is Full-Featured' }, {'id': 3, 'title': 'Tenjin is Easy-to-Use' }, ] context = { 'blog_post': blog_post, 'recent_posts': recent_posts, } ## render template import tenjin from tenjin.helpers import * from tenjin.helpers.html import text2html engine = tenjin.Engine(path=['views'], layout='_layout.pyhtml') html = engine.render('blog-post.pyhtml', context) print(html)
The result shows that captured string (with name 'sidebar') overwrites layout template content.
$ python main.py <html> <body> <div id="header-part"> <h1>My Great Blog</h1> </div> <div id="main-content"> <h2>Tenjin is Great</h2> <div class="blog-post"> Tenjin has great features.<br /> - Very Fast<br /> - Full Featured<br /> - Easy to Use<br /> </div> </div> <div id="sidebar-part"> <h3>Recent Posts</h3> <ul> <a href="/blog/1">Tenjin is Fast</a> <a href="/blog/2">Tenjin is Full-Featured</a> <a href="/blog/3">Tenjin is Easy-to-Use</a> </ul> </div> </body> </html>
Template Cache
Tenjin converts template file into Python script and save it as cache file. By default, it is saved as template-filename + '.cache' in bytecode format. You can change this behaviour by setting tenjin.Engine.cache
or passing cache object to tenjin.Engine
object.
For example, if you want to cache template object but want not to create '*.cache' file, use tenjin.MemoryCacheStorage
object.
import tenjin from tenjin.helpers import * ## change to store template cache into memory instead of file system tenjin.Engine.cache = tenjin.MemoryCacheStorage() engine = tenjin.Engine() ## or engine = tenjin.Engine(cache=tenjin.MemoryCacheStorage())
Fragment Cache
You can cache a certain part of HTML to improve performance. This is called as Fragment Cache.
<?py #@ARGS get_items ?> <div> <?py # fragment cache with key ('items/1') and lifetime (60sec) ?> <?py if not_cached('items/1', 60): ?> <ul> <?py for item in get_items(): ?> <li>${item}</li> <?py #endfor ?> </ul> <?py #endif ?> <?py echo_cached() ?> </div>
Tenjin stores fragments caches into memory by default. If you want to change or customize cache storage, see the following example.
import os, tenjin from tenjin.helpers import * ## create key-value store object if not os.path.isdir('cache.d'): os.mkdir('cache.d') kv_store = tenjin.FileBaseStore('cache.d') # file based ## set key-value store into tenjin.helpers.fagment_cache object tenjin.helpers.fragment_cache.store = kv_store ## context data ## (it is strongly recommended to create function object ## to provide pull-style context data) def get_items(): # called only when cache is expired return ['AAA', 'BBB', 'CCC'] context = {'get_items': get_items} ## render html engine = tenjin.Engine(path=['views']) html = engine.render('items.pyhtml', context) print(html)
$ python main.py <div> <ul> <li>AAA</li> <li>BBB</li> <li>CCC</li> </ul> </div>
You'll find that HTML fragment is cached into cache directory. This cache data will be expired at 60 seconds after.
$ cat cache.d/items/1 <ul> <li>AAA</li> <li>BBB</li> <li>CCC</li> </ul>
Logging
If you set logging object to tenjin.logger
, Tenjin will report loading template files.
For example:
import tenjin from tenjin.helpers import * ## set logging object import logging logging.basicConfig(level=logging.INFO) tenjin.logger = logging engine = tenjin.Engine() context = {'name': 'World'} html = engine.render('example.pyhtml', context)
If you run it first time, Tenjin will report that template object is stored into cache file.
$ python ex-logger.py > /dev/null INFO:root:[tenjin.MarshalCacheStorage] store cache (file='/home/user/example.pyhtml.cache')
And if you run it again, Tenjin will report that template object is loaded from cache file.
$ python ex-logger.py > /dev/null INFO:root:[tenjin.MarshalCacheStorage] load cache (file='/home/user/example.pyhtml.cache')
Using Tenjin with Google App Engine
Tenjin supports Google App Engine. All you have to do is just call tenjin.gae.init()
.
import tenjin from tenjin.helpers import * import tenjin.gae; tenjin.gae.init() ## it is recommended to configure logging import logging logging.basicConfig(level=logging.DEBUG) tenjin.logger = logging
tenjin.gae.init()
do the followings internally.
## change tenjin.Engine to cache template objects into memcache service ## (using CURRENT_VERSION_ID as namespace). ver = os.environ.get('CURRENT_VERSION_ID').split('.')[0] Engine.cache = tenjin.gae.GaeMemcacheCacheStorage(namespace=ver) ## change fragment cache store to use memcache service fragcache = tenjin.helpers.fragment_cache fragcache.store = tenjin.gae.aGaeMemcacheStore(namespace=ver) fragcache.lifetime = 60 # 1 minute fragcache.prefix = 'fragment.'
NOTICE: Google App Engine shares memcache in any version. In other words, Google App Engine doesn't allow to separate memcache for each version. This means that memcache data can be conflict between old and new version application in Google App Engine. Tenjin avoids this confliction by using version id as namespace.
Specify Function Names of escape() and to_str()
It is able to specify function names of escape()
and to_str()
which are used in converted Python script.
import tenjin from tenjin.helpers import * engine = tenjin.Engine(path=['views'], escapefunc='cgi.escape', tostrfunc='str') print(engine.get_template('page.pyhtml').script)
<p> escaped: ${value} not escaped: #{value} </p>
$ python main.py _buf.extend(('''<p> escaped: ''', cgi.escape(str(value)), ''' not escaped: ''', str(value), ''' </p>\n''', ));
If you need faster version of to_str()
and escape()
, see this section.
Preprocessing
Tenjin supports preprocessing of template. Preprocessing executes some logics when templates are loaded and that logics are not executed when rendering. Preprocessing makes your application much faster.
Basics of Preprocessing
Notation of preprocessing:
- <?PY ... ?>
- Preprocessing statement.
- #{{...}}
- Preprocessing expression (without HTML escape)
- ${{...}}
- Preprocessing expression (with HTML escape)
The following shows difference between ${ ... }
and ${{ ... }}
.
## normal expression value = ${value} ## with preprocessing value = ${{value}}
value = 'My Great Example' ## create engine object with preprocessing enabled import tenjin from tenjin.helpers import * engine = tenjin.Engine(path=['views'], preprocess=True) ## print Python script code print("------ converted script ------") print(engine.get_template('pp-example1.pyhtml').script) ## render html html = engine.render('pp-example1.pyhtml', {}) print("------ rendered html ------") print(html)
${{...}}
is evaluated at template converting stage.
$ python pp-example1.py ------ converted script ------ _buf.extend(('''## normal expression value = ''', escape(to_str(value)), ''' ## with preprocessing value = My Great Example\n''', )); ------ rendered html ------ ## normal expression value = My Great Example ## with preprocessing value = My Great Example
You can confirm preprocessed template by 'pytenjin -P
' command.
$ pytenjin -P -c 'value="My Great Example"' views/pp-example1.pyhtml ## normal expression value = ${value} ## with preprocessing value = My Great Example
If you want to see preprocessing script (not preprocessed script), use 'pytenjin -sP
' command.
$ pytenjin -sP -c 'value="My Great Example"' views/pp-example1.pyhtml _buf = []; _buf.extend(('''## normal expression value = ${value} ## with preprocessing value = ''', escape(to_str(_decode_params(value))), '''\n''', )); print(''.join(_buf))
Loop Expantion
It is possible to evaluate some logics by '<?PY ?>
' when convert template into Python script code. For example, you can expand loop in advance to improve performance.
<?PY states = { "CA": "California", ?> <?PY "NY": "New York", ?> <?PY "FL": "Florida", ?> <?PY "TX": "Texas", ?> <?PY "HI": "Hawaii", } ?> <?PY # ?> <?py chk = { params['state']: ' selected="selected"' } ?> <?PY codes = states.keys() ?> <?PY codes.sort() ?> <select name="state"> <option value="">-</option> <?PY for code in codes: ?> <option value="#{{code}}"#{chk.get('#{{code}}', '')}>${{states[code]}}</option> <?PY #endfor ?> </select>
Preprocessed script code shows that loop is expanded in advance. It means that loop is not executed when rendering template.
$ pytenjin -P views/pp-example2.pyhtml <?py chk = { params['state']: ' selected="selected"' } ?> <select name="state"> <option value="">-</option> <option value="CA"#{chk.get('CA', '')}>California</option> <option value="FL"#{chk.get('FL', '')}>Florida</option> <option value="HI"#{chk.get('HI', '')}>Hawaii</option> <option value="NY"#{chk.get('NY', '')}>New York</option> <option value="TX"#{chk.get('TX', '')}>Texas</option> </select>
Parameters
Assume that link_to() is a helper method which takes label and url and generate <a></a> tag. In this case, label and url can be parameterized by _p("...") and _P("..."). The former is converted into #{...} and the latter converted into ${...} by preprocessor.
<?PY ## ex. link_to('Show', '/show/1') => <a href="/show/1">Show</a> ?> <?PY def link_to(label, url): ?> <?PY import urllib ?> <?PY return '<a href="%s">%s</a>' % (urllib.quote(url), label) ?> <?PY # ?> #{{link_to('Show '+_P('params["name"]'), '/items/show/'+_p('params["id"]'))}}
The following shows that _P('...') and _p('...') are converted into ${...} and #{...} respectively.
$ pytenjin -P views/pp-example3.pyhtml <a href="/items/show/#{params["id"]}">Show ${params["name"]}</a>
There are many web-application framework and they provides helper functions. These helper functions are divided into two groups. link_to() or _() (function for M17N) return the same result when the same arguments are passed. These functions can be expanded by preprocessor. Some functions return the different result even if the same arguments are passed. These functions can't be expaned by preprocessor.
Preprocessor has the power to make view-layer much faster, but it may make the debugging difficult. You should use it carefully.
pytenjin
Command
See 'pytenjin -h
' for details.
Syntax Check
Command-line option '-z
' checks syntax of template files.
<ul> <?py for item in items: ?> <li>${item}</li> <?py #endfor ?> </ul>
$ pytenjin -z example.pyhtml example.pyhtml:4:3: unindent does not match any outer indentation level 4: #endfor ^
Error message is the same format as gcc compiler or java compiler. Error jump in Emacs or other editor is available.
Command-line option '-q' (quiet-mode) prints nothing if it has no errors.
Convert Template into Python Script
Command-line option '-s' converts template file into Python script code.
<ul> <?py for item in items: ?> <li>${item}</li> <?py #endfor ?> </ul>
$ pytenjin -s example.pyhtml _buf = []; _buf.extend(('''<ul>\n''', )); for item in items: _buf.extend((''' <li>''', escape(to_str(item)), '''</li>\n''', )); #endfor _buf.extend(('''</ul>\n''', )); print(''.join(_buf))
Option '-b' removes preamble ('_buf = []') and postamble ('print "".join(_buf)').
$ pytenjin -sb example.pyhtml _buf.extend(('''<ul>\n''', )); for item in items: _buf.extend((''' <li>''', escape(to_str(item)), '''</li>\n''', )); #endfor _buf.extend(('''</ul>\n''', ));
Retrieve Embedded Code
Tenjin allows you to retrieve embedded code from template files. This is aimed to help debugging template.
It is hard to debug large template files because HTML and embedded code are mixed in a file. Retrieving embedded code from template files will help you to debug large template files.
Assume the following template file.
<table> <?py i = 0 ?> <?py for item in items: ?> <?py i += 1 ?> <tr> <td>#{i}</td> <td>${item}</td> </tr> <?py #endfor ?> </table>
Option '-S' (or '-a retrieve') retrieves embedded codes.
$ pytenjin -Sb example.pyhtml i = 0 for item in items: i += 1 to_str(i); escape(to_str(item)); #endfor
Option '-X' (or '-a statements') retrieves only statements.
$ pytenjin -Xb example.pyhtml i = 0 for item in items: i += 1 #endfor
Option '-N' adds line numbers.
$ pytenjin -NXb example.pyhtml 1: 2: i = 0 3: for item in items: 4: i += 1 5: 6: 7: 8: 9: #endfor 10:
Option '-U' (unique) compress empty lines.
$ pytenjin -UNXb example.pyhtml 2: i = 0 3: for item in items: 4: i += 1 9: #endfor
Option '-C' (compact) removes empty lines.
$ pytenjin -CNXb example.pyhtml 2: i = 0 3: for item in items: 4: i += 1 9: #endfor
Execute Template File
You can execute template file in command-line.
<?py items = ['<AAA>', 'B&B', '"CCC"'] ?> <ul> <?py for item in items: ?> <li>${item}</li> <?py #endfor ?> </ul>
$ pytenjin example.pyhtml <ul> <li><AAA></li> <li>B&B</li> <li>"CCC"</li> </ul>
Context Data
You can specify context data with command-line option '-c
'.
<ul> <?py for item in items: ?> <li>${item}</li> <?py #endfor ?> </ul>
$ pytenjin -c 'items=["A","B","C"]' example.pyhtml <ul> <li>A</li> <li>B</li> <li>C</li> </ul>
If you want to specify several values, separate them by ';' such as '-c "x=10; y=20"'.
If you installed PyYAML library, you can specify context data in YAML format. Tenjin regards context data string as YAML format if it starts with '{'.
$ pytenjin -c '{items: [A, B, C]}' example.pyhtml <ul> <li>A</li> <li>B</li> <li>C</li> </ul>
In addition, Tenjin supports context data file in Python format or YAML format.
items = [ "AAA", 123, True, ]
$ pytenjin -f context.py example.pyhtml <ul> <li>AAA</li> <li>123</li> <li>True</li> </ul>
items: - AAA - 123 - true
$ pytenjin -f context.yaml example.pyhtml <ul> <li>AAA</li> <li>123</li> <li>True</li> </ul>
Trouble shooting
I got an SyntaxError exception.
Command-line option '-z' checks syntax of template file. You should check template by it.
File 'syntaxerr.pyhtml':
<?py for i in range(0, 10): ?> <?py if i % 2 == 0: ?> #{i} is even. <?py else ?> #{i} is odd. <?py #end ?> <?py #end ?>
Result:
$ pytenjin -z syntaxerr.pyhtml syntaxerr.pyhtml:4:9: invalid syntax 4: else ^
I got 'SyntaxError: encoding declaration in Unicode string'
This is because you added encoding declaration in template file AND you specified encoding option to tenjin.Engine() or tenjin.Template().
Solution:
- If you specify encoding option to tenjin.Engine() or tenjin.Template(), remove encoding declaration from your template file.
- If you want to add encoding declaration into your template file, don't specify encoding option to tenjin.Engine() or tenjin.Template().
I got UnicodeDecodeError, but I can't find what is wrong
If you got UnicodeDecodeError, you should do the following solutions.
- Set logger to
tenjin.logger
. If you set, Tenjin will report the content of_buf
.import logging logging.basicConfig(level=logging.DEBUG) tenjin.logger = logging
- Render tempalte with specifying
_buf
and check it directly._buf = [] try: engine.get_template("index.pyhtml").render(context, _buf=_buf) print(''.join(_buf)) except UnicodeDecodeError: for item in _buf: if isinstance(item, str): try: str.decode('ascii') except UnicodeDecodeError: print("*** failed to decode: %s" % repr(item))
Tips
Flexible Indentation
You may want Tenjin to be more flexible about indentation of statements. For example, You may not like such as:
<html> <body> <div id="main-content"> <?py if items: ?> <table> <tbody> <?py i = 0 ?> <?py for item in items: ?> <?py i += 1 ?> <tr> <td>${i}</td> <?py if item: ?> <td>${item}</td> <?py else: ?> <td>-</td> <?py #endif ?> </tr> <?py #endfor ?> </tbody> </table> <?py #endif ?> </div> </body> </html>
And you may prefer the following:
<html> <body> <div id="main-content"> <?py if items: ?> <table> <tbody> <?py i = 0 ?> <?py for item in items: ?> <?py i += 1 ?> <tr> <td>${i}</td> <?py if item: ?> <td>${item}</td> <?py else: ?> <td>-</td> <?py #endif ?> </tr> <?py #endfor ?> </tbody> </table> <?py #endif ?> </div> </body> </html>
If you want Tenjin to be more flexible about indentation, try http://gist.github.com/129297. Save this code as 'my_template.py' and try the following:
import tenjin from tenjin.helpers import * from my_template import MyTemplate tenjin.Engine.templateclass = MyTemplate import sys template_name = len(sys.argv) > 1 and sys.argv[1] or 'flexibleindent.pyhtml' engine = tenjin.Engine() print("-------------------- script") print(engine.get_template(template_name).script) print("-------------------- html") html = engine.render(template_name, {'items': ['AAA', None, 'CCC']}) print(html)
$ python flexibleindent.py flexibleindent.pyhtml -------------------- script _buf.extend(('''<html> <body> <div id="main-content">\n''', )); if items: _buf.extend((''' <table> <tbody>\n''', )); i = 0 for item in items: i += 1 _buf.extend((''' <tr> <td>''', escape(to_str(i)), '''</td>\n''', )); if item: _buf.extend((''' <td>''', escape(to_str(item)), '''</td>\n''', )); else: _buf.extend((''' <td>-</td>\n''', )); #endif _buf.extend((''' </tr>\n''', )); #endfor _buf.extend((''' </tbody> </table>\n''', )); #endif _buf.extend((''' </div> </body> </html>\n''', )); -------------------- html <html> <body> <div id="main-content"> <table> <tbody> <tr> <td>1</td> <td>AAA</td> </tr> <tr> <td>2</td> <td>-</td> </tr> <tr> <td>3</td> <td>CCC</td> </tr> </tbody> </table> </div> </body> </html>
NOTE: MyTemplate will be the default template class in the next release (1.0.0).
Webext
I have to say that bottleneck of Tenjin is calling to_str() and escape(), not string concatenation. Therefore if you want to make Tenjin much faster, you must make to_str() and escape() faster (or omit calling them).
For example, using str()
instead of to_str()
will make Tenjin much faster. The following benchmark result shows that str()
(= 'tenjin-str') is much faster than to_str()
(= 'tenjin').
str()
is faster than to_str()
$ cd Tenjin-X.X.X/benchmark $ python -V Python 2.5.5 $ python bench.py -q -n 10000 tenjin tenjin-str *** ntimes=10000 utime stime total real tenjin 3.7500 0.0400 3.7900 3.7936 tenjin-str 2.4500 0.0300 2.4800 2.4857
But str()
doesn't return empty string if argument is None
. In addition str()
raises UnicodeEncodeError frequently.
Other solution is Webext. Webext is an extension module which implement to_str()
and escape()
in C language.
import tenjin from tenjin.helpers import * from webext import to_str, escape # use webext's functions instead of tenjin's
Benchmark script in Tenjin already supports Webext. It shows that Webext makes Tenjin much faster especially html escaping.
### without html escaping $ cd Tenjin-X.X.X/ $ cd benchmark/ $ python bench.py -q -n 10000 tenjin tenjin-str tenjin-webext *** ntimes=10000 utime stime total real tenjin 3.8100 0.0400 3.8500 3.8462 tenjin-str 2.4500 0.0200 2.4700 2.4815 tenjin-webext 2.4500 0.0300 2.4800 2.4825 ## with html escaping $ python bench.py -e -q -n 10000 tenjin tenjin-str tenjin-webext *** ntimes=10000 utime stime total real tenjin 7.2900 0.0500 7.3400 7.4669 tenjin-str 5.7400 0.0400 5.7800 5.8202 tenjin-webext 2.9700 0.0400 3.0100 3.0079
M17N Page
If you have M17N site, you can make your site faster by Tenjin.
In M17N-ed site, message translation function (such as _('message-key')
) is called many times. Therefore if you can eliminate calling that function, your site can be faster. And Tenjin can do it by preprocessing.
The point is:
- Change cache filename according to language. For example, create cache file 'file.pyhtml.en.cache', 'file.pyhtml.fr.cache', 'file.pyhtml.it.cache', and so on from a template file 'file.pyhtml'. This can be done by overriding CacheStorage#_cachename().
- Create CacheStorage and Engine object for each language.
- Enable preprocessing to create different cache content for each language.
The following is an example to generate M17N pages from a template file.
<div> <?PY ## '_()' represents translator method ?> <p>${{_('Hello')}} ${username}!</p> </div>
# -*- coding: utf-8 -*- import tenjin from tenjin.helpers import * import re ## ## message catalog to translate message ## MESSAGE_CATALOG = { 'en': { 'Hello': 'Hello', 'Good bye': 'Good bye', }, 'fr': { 'Hello': 'Bonjour', 'Good bye': 'Au revoir', }, } ## ## create translation function and return it. ## ex. ## _ = create_m18n_func('fr') ## print _('Hello') #=> 'Bonjour' ## def create_m18n_func(lang): dct = MESSAGE_CATALOG.get(lang) if not dct: raise ValueError("%s: unknown lang." % lang) def _(message_key): return dct.get(message_key) return _ # or return dct.get ## ## cache storage class to cache template object for each language ## class M17nCacheStorage(tenjin.MarshalCacheStorage): lang = 'en' # default language def __init__(self, *args, **kwargs): if 'lang' in kwargs: lang = kwargs.pop('lang') if lang: self.lang = lang tenjin.MarshalCacheStorage.__init__(self, *args, **kwargs) ## change cache filename to 'file.pyhtml.lang.cache' def _cachename(self, fullpath): return "%s.%s.cache" % (fullpath, self.lang) ## ## test program ## if __name__ == '__main__': ## create cache storage and engine for English m17ncache = M17nCacheStorage(lang='en') engine_en = tenjin.Engine(preprocess=True, cache=m17ncache) ## render html for English context = { 'username': 'World' } context['_'] = create_m18n_func('en') html = engine_en.render('m18n.pyhtml', context) print("--- lang: en ---") print(html) ## create cache storage and engine for French m17ncache = M17nCacheStorage(lang='fr') engine_fr = tenjin.Engine(preprocess=True, cache=m17ncache) ## render html for French context = { 'username': 'World' } context['_'] = create_m18n_func('fr') html = engine_fr.render('m18n.pyhtml', context) print("--- lang: fr ---") print(html)
$ python m18n.py --- lang: en --- <div> <p>Hello World!</p> </div> --- lang: fr --- <div> <p>Bonjour World!</p> </div>
After that, you can find two cache files are created.
$ ls m18n.pyhtml* m18n.pyhtml m18n.pyhtml.en.cache m18n.pyhtml.fr.cache
And each cache files have different content.
### "_('Hello')" is translated into "Hello" in Engilish cache file $ pytenjin -a dump m18n.pyhtml.en.cache _buf.extend(('''<div> <p>Hello ''', escape(to_str(username)), '''!</p> </div>\n''', )); ### "_('Hello')" is translated into "Bonjour" in French cache file $ pytenjin -a dump m18n.pyhtml.fr.cache _buf.extend(('''<div> <p>Bonjour ''', escape(to_str(username)), '''!</p> </div>\n''', ));
Template Inheritance
Tenjin doesn't support Template Inheritance which Django template engine does. But you can emulate it by capturing(*2). See this section for details.
- (*2)
- Notice that capturing is useful but not so powerful than template inheritance.