Module `deepsport_utilities.dataset.common`

Global variables

var SUBSETS_CONVERTION: Group dict items

Functions

def group(d, groups)

Expand source code

def group(d, groups):
    return {name: {key: d[key] for key in group} for name, group in groups.items()}

def transforms_wrapper(subsets, transforms)

Expand source code

def transforms_wrapper(subsets, transforms):
    return {
        "training": DataAugmentationDataset(subsets.pop("training"), transforms, Stage.TRAIN),
           **{name: DataAugmentationDataset(subset, transforms) for name, subset in subsets.items()}
    }

Classes

class DataAugmentationDataset (dataset, transforms, stage=Stage.EVAL)

Expand source code

class DataAugmentationDataset(TransformedDataset):
    def __init__(self, dataset, transforms, stage=Stage.EVAL):
        super().__init__(dataset, transforms)
        self.stage = stage

    def query_item(self, key):
        item = self.parent.query_item(key)
        for transform in self.transforms:
            signature = inspect.signature(transform.__call__)
            if len(signature.parameters) == 3:
                item = transform(key, item, self.stage)
            elif len(signature.parameters) == 2:
                item = transform(key, item)
        return item

A dataset that passes yielded items through transforms

>>> d = DictDataset({0: ("Denzel", "Washington"), 1: ("Tom", "Hanks")})
>>> d = TransformedDataset(d, [lambda x: (x[0][:1]+".", x[1])])
>>> d.query([0, 1])
(array(['D.', 'T.'], ...), array(['Washington', 'Hanks'], ...))

Creates a dataset performing operations for modifying another

Ancestors

mlworkflow.datasets.TransformedDataset
mlworkflow.datasets.Dataset

Methods

def query_item(self, key)

Expand source code

def query_item(self, key):
    item = self.parent.query_item(key)
    for transform in self.transforms:
        signature = inspect.signature(transform.__call__)
        if len(signature.parameters) == 3:
            item = transform(key, item, self.stage)
        elif len(signature.parameters) == 2:
            item = transform(key, item)
    return item

Returns a tuple for one item, typically (Xi, Yi), or (Xi,)

class Stage (value, names=None, *, module=None, qualname=None, type=None, start=1)

Expand source code

class Stage(IntFlag):
    TRAIN = 1
    EVAL  = 2

An enumeration.

Ancestors

enum.IntFlag
builtins.int
enum.Flag
enum.Enum

Class variables

var EVAL
var TRAIN