Previous topic

pydoop.hdfs — HDFS API

Next topic

pydoop.jc — Pydoop Script Configuration Access

Get Pydoop

Contributors

Pydoop is developed by: CRS4

And generously hosted by: Get Pydoop at SourceForge.net. Fast, secure and Free Open Source software downloads

pydoop.utils — Utility Functions

This module contains general utility functions for application writing.

pydoop.utils.jc_configure(obj, jc, k, f, df=None)

Gets a configuration parameter from jc and automatically sets a corresponding attribute on obj.

Parameters:
  • obj (any object, typically a MapReduce component) – object on which the attribute must be set
  • jc (JobConf) – a job configuration object
  • k (string) – a configuration key
  • f (string) – name of the attribute to set
  • df (string) – default value for the attribute if k is not present in jc
pydoop.utils.jc_configure_bool(obj, jc, k, f, df=None)

Works like jc_configure(), but converts jc[k] to a boolean.

pydoop.utils.jc_configure_float(obj, jc, k, f, df=None)

Works like jc_configure(), but converts jc[k] to a float.

pydoop.utils.jc_configure_int(obj, jc, k, f, df=None)

Works like jc_configure() , but converts jc[k] to an integer.

pydoop.utils.jc_configure_log_level(obj, jc, k, f, df=None)

Works like jc_configure(), but converts jc[k] to a logging level.

The default value, if specified, must be a log level string, e.g., ‘INFO’.

pydoop.utils.make_input_split(filename, offset, length)

Build a fake (i.e., not tied to a real file) InputSplit. This is used for testing.

Parameters:
  • filename (string) – file name
  • offset (int) – byte offset of the split with respect to the beginning of the file
  • length (int) – length of the split in bytes
pydoop.utils.raise_pydoop_exception(msg)

Raise a generic Pydoop exception.

The exception is built at the C++ level and translated into a UserWarning by the Boost wrapper.