DataSet

class escape.DataSet(raw_datasets: dict = None, alias_mappings: dict = None, results_file=None, mode='r', perm=None, name=None, lazy_loading=False)

Bases: object

__init__(raw_datasets: dict = None, alias_mappings: dict = None, results_file=None, mode='r', perm=None, name=None, lazy_loading=False)

append(data, auto_format=True, as_hickle=False, as_pickle=False, as_datastorage=False, esc_type=None, name=None)

close()

compute_datasets_max_element_size(max_element_size=5000, verbose=0, **kwargs)

classmethod create_with_new_result_file(results_filepath, mode='w', force_overwrite=False, name=None)

get_array(spec, computed=True)

get_datasets_max_element_size(max_element_size=5000, verbose=0)

get_structure_tree(base=None)

classmethod load_from_result_file(results_filepath, lazy_loading=False, name=None, perm=None)

store_datasets_max_element_size(max_element_size=5000, lock='auto', verbose=0, **kwargs)

escape.storage.dataset.merge_datasets(datasets, only_escape_arrays=False, **kwargs_dataset)

Merges datasets of multiple dataset containers into one dataset container. escape.arrays are here concatenated to a merged escape_array, other datatypes are only merges as python lists.

Args:

datasets list or iterable of the dataset containers to be merged (type DataSet). only_escape_arrays (bool, optional): optionally only merging of escape.arrays. Defaults to False.

Returns:

dataset_merged: new DataSet instance where the data have been merged into.

escape.storage.dataset.convert_resultsfile(filename, out_filename=None, out_directory=None, out_type='h5', force_overwrite=False, close_if_feasible=True)

Convert resultsfile, typically from zarr to h5 for easier handling.

Args:

filename (string): Input file name out_filename (string, optional): output filename, if None is deduced from input name. Defaults to None. out_directory (string or Path instance, optional): output directory. Defaults to None. out_type (str, optional): output file type. Defaults to ‘h5’. force_overwrite (bool, optional): Fore overwriting if output file exists. Defaults to False. close_if_feasible (bool, optional): close the covertet file (for h5). Defaults to True.

Returns:

str or dataset: returns output filename in case file is closed otherwise the output dataset.