module documentation

repositoryutil.py - ZMS repository utility module.

This module provides helper functions and classes for repository access from Python scripts and other ZMS internals.

License: GNU General Public License v2 or later, Organization: ZMS Publishing

Function create_modelfileset Mid-level: Build the complete set of file records for one model object from its ZODB representation.
Function get_class Get class from py-string.
Function get_diffs Undocumented
Function get_file_from_disk Mid-level: Read a single file from disk and return a metadata record.
Function get_init_py Generate a Python class representation of a given object.
Function get_init_yaml Serialize a Python object into a YAML-formatted string.
Function get_modelfileset_from_disk High-level: Collect all model files from a filesystem repository path.
Function get_modelfileset_from_zodb High-level: Export model files from ZODB via a repository provider.
Function get_models_from_disk High-level: Load and parse all model definitions from a filesystem repository path.
Function get_providers Returns list of repository-providers.
Function get_system_conf_basepath Returns system conf-basepath.
Function parse_modelfile Low-level: Parse in-memory model file contents into a model definition dict.
Function read_file_from_disk Low-level: Read raw file contents from the filesystem.
Function value_data Helper function to resolve embedded attribute data from sibling resource files.
Variable security Undocumented
Variable yamlutil Undocumented
def create_modelfileset(o, init_files): (source)

Mid-level: Build the complete set of file records for one model object from its ZODB representation.

Given a ZODB model object dict o and its pre-serialized init file content (from get_init_py or get_init_yaml), this function produces the full filename-to-metadata mapping that represents the object on disk. The mapping includes:

  • The init file itself (__init__.py or __init__.yaml).
  • Sibling resource files (templates, scripts, images) extracted from the object's attribute lists.
  • A readme.md sibling if the object carries a readme field.

Does not perform filesystem I/O — it operates purely on in-memory data. Called by get_modelfileset_from_zodb for each model object.

Parameters
o:dictRepository object dict. Expected keys include id, acquired, __filename__, __icon__, __description__, revision, and capitalized attribute-list keys.
init_files:dictPre-serialized init content grouped by format, e.g. {'py': [line, ...]} or {'yaml': [line, ...]}.
Returns
dictMapping from generated filenames to file-record dicts, each containing id, filename, data, version, meta_type.
See Also
get_modelfileset_from_zodb, get_init_py, get_init_yaml
def get_class(py): (source)

Get class from py-string.

This function extracts the class definition from a given Python code string and executes it to return the defined class object. It uses regular expressions to identify the class name and then executes the code to obtain the class object.

Parameters
py:strA string containing Python code that defines a class.
Returns
typeThe class object defined in the input string.
def get_diffs(local, remote, ignore=True): (source)

Undocumented

def get_file_from_disk(self, base, path, name, id=None): (source)

Mid-level: Read a single file from disk and return a metadata record.

Delegates to read_file_from_disk for the actual I/O, then packages the result into a dict with id, filename (relative to base), data, and version (file modification time).

Called by get_modelfileset_from_disk.

Parameters
selfUndocumented
base:strThe repository base path used to compute relative filenames.
path:strThe directory containing the file.
name:strThe filename to read.
idUndocumented
Returns
dictMetadata record {id, filename, data, version}.
See Also
read_file_from_disk
def get_init_py(self, o): (source)

Generate a Python class representation of a given object.

This function takes an object `o` (typically a dictionary) and generates a Python class definition as a list of strings. The generated class includes attributes and nested classes based on the structure and content of the input object.

Parameters
self:objectThe object context.
o:dictInput object containing keys and values to be represented as a Python class.
Returns
listPython class definition lines.
def get_init_yaml(self, o): (source)

Serialize a Python object into a YAML-formatted string.

This method processes a given object `o` and converts it into a YAML representation. It handles attributes and keys in the object, ensuring that non-serializable elements (e.g., Acquisition-Wrappers) are excluded.

Parameters
self:objectThe object context.
o:dictInput dictionary-like object to be serialized.
Returns
strYAML-formatted string representing the serialized object.
def get_modelfileset_from_disk(self, basepath, deep=True): (source)

High-level: Collect all model files from a filesystem repository path.

Recursively traverses the directory tree rooted at basepath looking for repository object folders (identified by __init__.py or __init__.yaml). For each folder found, the init file and all sibling resource files are read into metadata records.

Delegates to:

  • get_file_from_disk — to read each individual file and build its metadata record {id, filename, data, version}.
  • parse_modelfile — to extract the revision from the init file's parsed object definition.
Parameters
selfUndocumented
basepath:strThe root filesystem path to scan for repository objects. Must be an existing directory.
deep:boolIf True, recursively traverse subdirectories. If False, only scan the root basepath directory. Defaults to True.
Returns
dict

Dictionary mapping relative file paths to metadata dictionaries. Each metadata dictionary contains:

  • id: Object identifier from repository definition or filename
  • filename: Relative path from basepath
  • data: File content as string (for .yaml/.py) or bytes
  • version: Revision from repository definition or file modification time
Raises
IOErrorIf file read operations fail during traversal.
See Also
get_file_from_disk, parse_modelfile
def get_modelfileset_from_zodb(self, provider, ids=None): (source)

High-level: Export model files from ZODB via a repository provider.

Queries the provider for repository objects stored in the ZODB, then serializes each object into a set of filesystem-ready file records (init file + sibling resource files).

This is the ZODB counterpart to get_modelfileset_from_disk; both return the same {filename: {{id, filename, data, version, ...}}} dict structure so that results can be compared with get_diffs.

Delegates to:

  • provider.provideRepository(ids) — to retrieve objects from ZODB.
  • get_init_py / get_init_yaml — to serialize objects into init file content (Python class or YAML).
  • create_modelfileset — to build the filename-to-metadata mapping from the serialized init content and sibling resource objects.
Parameters
selfUndocumented
provider:IZMSRepositoryProviderA repository provider implementing IZMSRepositoryProvider.provideRepository().
ids:list or NoneOptional list of object IDs to export. If None, all objects from the provider are exported.
Returns
dictDictionary mapping filenames to metadata records.
See Also
get_modelfileset_from_disk, create_modelfileset, get_init_py, get_init_yaml
def get_models_from_disk(self, basepath, deep=True): (source)

High-level: Load and parse all model definitions from a filesystem repository path.

Traverses the directory tree rooted at basepath, reads each init file (__init__.py / __init__.yaml), parses it into a structured model definition, and resolves embedded attribute data from sibling resource files.

Unlike get_modelfileset_from_disk (which returns raw file records), this function returns fully parsed model objects keyed by object ID.

Delegates to:

Parameters
selfUndocumented
basepath:strThe root filesystem path from which to read the repository. Must be an existing directory.
deep:boolIf True, recursively traverse subdirectories. If False, only process the top-level directory (level 0). Defaults to True.
Returns
dict

Dictionary mapping repository object IDs to their parsed metadata. Each entry contains:

  • For Python objects: class attributes (excluding those starting with '__') with nested class definitions converted to sorted lists of attribute dicts
  • For YAML objects: parsed YAML structure with resolved 'id' field
  • Both formats: 'data' field populated for sibling resource files matching object IDs, and 'readme' field if readme.md exists in the directory
See Also
get_modelfileset_from_disk, read_file_from_disk, parse_modelfile, value_data
def get_providers(self): (source)

Returns list of repository-providers.

These are collected by traversing the ZMS object tree starting from the document element and checking for objects that provide the IZMSRepositoryProvider interface. The traversal also includes objects that provide the IZMSConfigurationProvider interface, allowing for nested configurations that may contain additional repository providers.

Parameters
self:objectThe object context.
Returns
listList of repository-providers.
def get_system_conf_basepath(): (source)

Returns system conf-basepath.

def parse_modelfile(self, path, filename, data): (source)

Low-level: Parse in-memory model file contents into a model definition dict.

Interprets the data string (previously read by read_file_from_disk) according to the file extension:

  • .yaml: parsed via yamlutil.parse into a dict keyed by object ID.
  • .py: executed via get_class to extract the class __dict__.

Does not perform any filesystem I/O — it operates solely on the data parameter.

Called by get_modelfileset_from_disk and get_models_from_disk.

Parameters
selfUndocumented
path:strDirectory path (used only for error messages).
filename:strFilename including extension (determines parse strategy).
data:strThe file contents to parse (text string).
Returns
dictParsed model definition dict; on error contains a revision key with the error message.
See Also
read_file_from_disk, get_class, yamlutil.parse
def read_file_from_disk(self, path, filename): (source)

Low-level: Read raw file contents from the filesystem.

Opens the file at path/filename in binary mode and attempts UTF-8 decoding. Returns str if decodable, raw bytes otherwise, or None if the file does not exist.

This is the lowest-level I/O helper in the repository utilities. Called by get_file_from_disk, get_models_from_disk, value_data, and any other function needing raw file contents.

Parameters
selfUndocumented
path:strDirectory containing the file.
filename:strName of the file to read.
Returns
str or bytes or NoneFile contents as str (UTF-8) or bytes, or None if the file does not exist.
def value_data(self, vv, names, path, file): (source)

Helper function to resolve embedded attribute data from sibling resource files.

For a given attribute value vv (which may be a dict representing an object reference), this function checks if it contains an 'id' field that matches any of the sibling files in the same directory. If a match is found, it reads the corresponding file content and assigns it to the 'data' field of vv. This allows repository definitions to reference external resource files by ID, which are then loaded and included in the final model representation.

Parameters
self:objectThe object context.
vv:anyThe attribute value to check for embedded data references.
names:listList of sibling filenames in the same directory as the repository object.
path:strThe directory path where the repository object is located.
file:strThe filename of the repository object's init file (used for error messages).
Returns
anyThe original attribute value vv with an added 'data' field if a matching file was found and read.
security = (source)

Undocumented

yamlutil = (source)

Undocumented