Package ZestyParser :: Module Tokens
[show private | hide private]
[frames | no frames]

Module ZestyParser.Tokens


Version: 0.7.0

Author: Adam Atlas

Contact: adam@atlas.st

Copyright: Copyright 2006-2007 Adam Atlas. Released under the terms of the GNU General Public License.

Classes
AbstractToken Base class from which most tokens defined in this module derive.
TokenWrapper If you write your own token type in a way other than subclassing AbstractToken, e.g.
_EOF Matches returning None if the parser is at the end of its input.
    Basic Tokens
RawToken A class whose instances match only a particular string.
Token A class whose instances match Python regular expressions.
TakeToken A class whose instances match and return a given number of characters from the parser's data.
    Complex Tokens
CompositeToken A class whose instances match any of a number of tokens.
TokenSequence A class whose instances match a sequence of tokens.
TokenSeries A particularly versatile class whose instances match one token multiple times (with a great degree of customizability).
    Special Tokens
Defer A token which takes a callable (generally a lambda) which takes no arguments and itself returns a token.
Default A class whose instances always return desc and do not advance the parser's cursor.
Lookahead Scans for another token and returns its result as usual, but doesn't actually advance the parser's cursor.
Negative Scans for another token, and only matches (returning True) if that token did not match.
    TokenSequence Flags
Omit See TokenSequence.
Skip See TokenSequence.
Only See TokenSequence.

Function Summary
  Const(value)
This is a factory function used for when you want an AbstractToken-compatible callback that returns a constant.
  count_args(callable)

Variable Summary
Default EmptyToken: A Default instance initialized with the empty string.
_EOF EOF: A token which matches (and returns None) if the parser is at the end of its data sequence.
list rstack = []

Function Details

Const(value)

This is a factory function used for when you want an AbstractToken-compatible callback that returns a constant. Example:
   RawToken('foo') >> Const('bar')
This token matches 'foo' as usual, but always returns 'bar'.

Variable Details

EmptyToken

A Default instance initialized with the empty string.
Type:
Default
Value:
Default ''                                                             

EOF

A token which matches (and returns None) if the parser is at the end of its data sequence.

In ZestyParser, a token object must, at minimum, be a callable taking a ZestyParser instance and its current cursor as parameters. It can do whatever it needs with the parser's data and cursor properties before returning. It may raise NotMatched to indicate to the ZestyParser instance that it failed to match; it may also raise ParseError to indicate, for instance, that it began matching successfully but encountered an unrecoverable error.

The Tokens module contains a variety of predefined token classes (instances of which are callable) and other valid token objects which should cover most parsing situations.
Type:
_EOF
Value:
_EOF                                                                   

rstack

Type:
list
Value:
[]                                                                     

Generated by Epydoc 2.1 on Thu Mar 22 02:13:39 2007 http://epydoc.sf.net