Package ZestyParser :: Module Tokens :: Class AbstractToken
[show private | hide private]
[frames | no frames]

Class AbstractToken

Known Subclasses:
CompositeToken, Default, Defer, Omit, RawToken, Skip, TakeToken, Token, TokenSequence, TokenSeries, _EOF

Base class from which most tokens defined in this module derive. Subclassing this is not required for writing tokens, since they can be any callable with certain semantics, but this class provides several useful services for creating reusable token classes, such as callback support and convenient operator overloading.
Method Summary
  __init__(self, desc, callback, as, name)
  __add__(self, other)
Allows you to construct TokenSequences with the + operator.
  __or__(self, other)
Allows you to construct CompositeTokens with the | operator.
  __repr__(self)
  __rshift__(self, callback)
Convenience overloading for setting the callback of a token whose initializer you do not call directly, such as the result of combining tokens with + or |.
  __xor__(self, message)
Overloading for setting the failMessage of a token.
  preprocessResult(self, parser, data, origCursor)
Method called by subclasses' __call__ methods to add uniform support for the callback and as parameters.

Instance Variable Summary
  as: An optional callable which, if not None, will be called in the same manner as a callback (after any callback and before returning to the parser instance), but will be passed only one argument: the data matched (or returned by the callback, if any).
  callback: An optional callable which, if not None, will be called whenever an instance matches successfully.
  desc: The generic "description" variable which stores the "essence" of any given instance.

Class Variable Summary
NoneType failMessage = None                                                                  

Method Details

__add__(self, other)
(Addition operator)

Allows you to construct TokenSequences with the + operator.

__or__(self, other)
(Or operator)

Allows you to construct CompositeTokens with the | operator.

__rshift__(self, callback)

Convenience overloading for setting the callback of a token whose initializer you do not call directly, such as the result of combining tokens with + or |.
Parameters:
callback - An AbstractToken-compatible callback.
           (type=callable)
Returns:
A copy of self with the callback property set to callback.

__xor__(self, message)

Overloading for setting the failMessage of a token.
Parameters:
message - The message to be raised with ParseError if this token fails to match.
           (type=str)
Returns:
A copy of self with the failMessage property set to callback.

preprocessResult(self, parser, data, origCursor)

Method called by subclasses' __call__ methods to add uniform support for the callback and as parameters. Pass your __call__ method's return value through this if you're subclassing AbstractToken yourself.

Instance Variable Details

as

An optional callable which, if not None, will be called in the same manner as a callback (after any callback and before returning to the parser instance), but will be passed only one argument: the data matched (or returned by the callback, if any). Its main purpose is to allow you to concisely do things like Token('[0-9]+', group=0, as=int) -- the builtin callable int will be passed the text matched by the regex, so the token will ultimately return an integer instead of a string or a regex match object. You can also use this property with AHT types, for more complex multi-stage parsing. See the n3.py and n3rdflib.py examples for a demonstration of this.

callback

An optional callable which, if not None, will be called whenever an instance matches successfully. It may take one, two, or three parameters, depending on its needs. If one, it will be passed whatever data the token matched (i.e. whatever it would normally have returned upon being called). If two, it will be passed the ZestyParser instance and the data. If three, it will be passed the parser, the data, and the what the parser's cursor was when this token started matching. Callbacks may raise NotMatched or ParseError with the usual behaviour. They should also return a value, which will be returned to the calling ZestyParser instance.

desc

The generic "description" variable which stores the "essence" of any given instance. Subclasses use this as needed.

Class Variable Details

failMessage

Type:
NoneType
Value:
None                                                                  

Generated by Epydoc 2.1 on Thu Jan 11 23:26:09 2007 http://epydoc.sf.net