Module src.PyOghma_ML.Training
This module provides functionality for training machine learning models using TensorFlow/Keras.
It includes classes and utilities for configuring, training, and saving models. The module supports various model architectures, including residual models and multi-input models, and allows for customization of hyperparameters such as learning rates, activation functions, and layer configurations.
Classes
class Model_Settings (initializer: str = 'he_normal',
activation: str = 'silu',
regularization=None,
layer_nodes: list = None,
dropout: list = None,
batch_norm: bool = True,
epochs: int = 256,
inital_learning_rate: float = 1e-06,
gamma_learning_rate: float = 0.001,
power_learning_rate: float = 3,
batch_size: int = 2024,
patience: int = 16,
loss_function: str = 'mse',
metrics: list = None,
training_percentage: float = 0.8,
validation_percentage: float = 0.2,
permutations_limit: int = 1000,
ensemble_presample: int = 1,
ensemble_maximum: int = 2,
ensemble_tollerance: float = 0.001,
ensemble_patience: int = 10,
inputs: int = 1,
decay_rate: int = 32)-
Expand source code
class Model_Settings: """ A comprehensive configuration class for neural network model settings. This class encapsulates all hyperparameters and configuration options needed for training machine learning models in the PyOghma_ML framework. It provides sensible defaults while allowing full customization of the training process. Architecture Settings: - Layer configuration (nodes, dropout, normalization) - Activation functions and weight initialization - Regularization techniques Training Parameters: - Learning rate scheduling with exponential decay - Batch size and epoch configuration - Early stopping with patience Ensemble Configuration: - Multiple model training and aggregation - Performance tolerance and optimization limits - Data sampling strategies Attributes: initializer (str): Weight initialization method for model layers. Common options: 'he_normal', 'glorot_uniform', 'random_normal'. activation (str): Activation function for hidden layers. Recommended: 'silu', 'relu', 'tanh', 'gelu'. regularization: Regularization technique (L1, L2, or custom). layer_nodes (list): Number of neurons in each hidden layer. dropout (list): Dropout rates for each layer (prevents overfitting). batch_norm (bool): Whether to apply batch normalization. epochs (int): Maximum number of training epochs. inital_learning_rate (float): Starting learning rate for optimization. gamma_learning_rate (float): Decay factor for learning rate schedule. power_learning_rate (float): Power parameter for learning rate decay. batch_size (int): Number of samples per training batch. patience (int): Early stopping patience (epochs without improvement). loss_function (str): Loss function for training ('mse', 'mae', etc.). metrics (list): Additional metrics to monitor during training. training_percentage (float): Fraction of data used for training. validation_percentage (float): Fraction of data used for validation. permutations_limit (int): Maximum permutations for data augmentation. ensemble_presample (int): Number of models to pre-train for ensemble. ensemble_maximum (int): Maximum number of models in ensemble. ensemble_tollerance (float): Performance tolerance for ensemble inclusion. ensemble_patience (int): Patience for ensemble optimization. inputs (int): Number of input branches for multi-input models. decay_rate (float): Additional decay parameter for learning rate. Example: >>> settings = Model_Settings( ... layer_nodes=[128, 64, 32], ... dropout=[0.2, 0.3, 0.4], ... epochs=500, ... batch_size=1024 ... ) """ def __init__(self, initializer: str = 'he_normal', activation: str = 'silu', regularization = None, layer_nodes: list = None, dropout: list = None, batch_norm: bool = True, epochs: int = 256, inital_learning_rate: float = 1e-6, gamma_learning_rate: float = 0.001, power_learning_rate: float = 3, batch_size: int = 2024, patience: int = 16, loss_function: str = 'mse', metrics: list = None, training_percentage: float = 0.8, validation_percentage: float = 0.2, permutations_limit: int = 1000, ensemble_presample: int = 1, ensemble_maximum: int = 2, ensemble_tollerance: float = 1e-3, ensemble_patience: int = 10, inputs: int = 1, decay_rate: int = 32) -> None: self.initializer = initializer self.activation = activation self.regularization = regularization self.layer_nodes = layer_nodes if layer_nodes is not None else [128, 128, 128, 128] self.dropout = dropout if dropout is not None else [0.01, 0.01, 0.01, 0.1] self.batch_norm = batch_norm self.epochs = epochs self.inital_learning_rate = inital_learning_rate self.gamma_learning_rate = gamma_learning_rate self.power_learning_rate = power_learning_rate self.batch_size = batch_size self.patience = patience self.loss_function = loss_function self.metrics = metrics if metrics is not None else [tf.keras.metrics.MeanAbsoluteError()] self.training_percentage = training_percentage self.validation_percentage = validation_percentage self.permutations_limit = permutations_limit self.ensemble_presample = ensemble_presample self.ensemble_maximum = ensemble_maximum self.ensemble_tollerance = ensemble_tollerance self.ensemble_patience = ensemble_patience self.inputs = inputs self.decay_rate = decay_rate
A comprehensive configuration class for neural network model settings.
This class encapsulates all hyperparameters and configuration options needed for training machine learning models in the PyOghma_ML framework. It provides sensible defaults while allowing full customization of the training process.
Architecture Settings: - Layer configuration (nodes, dropout, normalization) - Activation functions and weight initialization - Regularization techniques
Training Parameters: - Learning rate scheduling with exponential decay - Batch size and epoch configuration - Early stopping with patience
Ensemble Configuration: - Multiple model training and aggregation - Performance tolerance and optimization limits - Data sampling strategies
Attributes
initializer
:str
- Weight initialization method for model layers. Common options: 'he_normal', 'glorot_uniform', 'random_normal'.
activation
:str
- Activation function for hidden layers. Recommended: 'silu', 'relu', 'tanh', 'gelu'.
regularization
- Regularization technique (L1, L2, or custom).
layer_nodes
:list
- Number of neurons in each hidden layer.
dropout
:list
- Dropout rates for each layer (prevents overfitting).
batch_norm
:bool
- Whether to apply batch normalization.
epochs
:int
- Maximum number of training epochs.
inital_learning_rate
:float
- Starting learning rate for optimization.
gamma_learning_rate
:float
- Decay factor for learning rate schedule.
power_learning_rate
:float
- Power parameter for learning rate decay.
batch_size
:int
- Number of samples per training batch.
patience
:int
- Early stopping patience (epochs without improvement).
loss_function
:str
- Loss function for training ('mse', 'mae', etc.).
metrics
:list
- Additional metrics to monitor during training.
training_percentage
:float
- Fraction of data used for training.
validation_percentage
:float
- Fraction of data used for validation.
permutations_limit
:int
- Maximum permutations for data augmentation.
ensemble_presample
:int
- Number of models to pre-train for ensemble.
ensemble_maximum
:int
- Maximum number of models in ensemble.
ensemble_tollerance
:float
- Performance tolerance for ensemble inclusion.
ensemble_patience
:int
- Patience for ensemble optimization.
inputs
:int
- Number of input branches for multi-input models.
decay_rate
:float
- Additional decay parameter for learning rate.
Example
>>> settings = Model_Settings( ... layer_nodes=[128, 64, 32], ... dropout=[0.2, 0.3, 0.4], ... epochs=500, ... batch_size=1024 ... )
class Training (training_features: numpy.ndarray,
training_targets: numpy.ndarray,
validation_features: numpy.ndarray,
validation_targets: numpy.ndarray,
dir: str,
model_settings:| None = None,
model_type: str | None = None,
existing: bool | None = False)-
Expand source code
class Training: """ A class for training machine learning models using TensorFlow/Keras. This class supports different model architectures, including residual models, and provides methods for training, saving, and configuring models. """ def __init__(self, training_features: np.ndarray, training_targets: np.ndarray, validation_features: np.ndarray, validation_targets: np.ndarray, dir: str, model_settings: Optional[dataclass] = None, model_type: Optional[str] = None, existing: Optional[bool] = False) -> None: """ Initialize a Training instance and train the model. This method sets up and trains a neural network model using the provided training and validation data. It supports both creating new models and continuing training from existing models. Args: training_features (numpy.ndarray): Input features for training the model. training_targets (numpy.ndarray): Target values for training the model. validation_features (numpy.ndarray): Input features for model validation. validation_targets (numpy.ndarray): Target values for model validation. dir (str): Directory path where the trained model will be saved. model_settings (Model_Settings, optional): Configuration settings for the model including architecture, hyperparameters, and training parameters. model_type (str, optional): Type of model architecture to use. Supported types include 'Residual' and default Sequential models. existing (bool, optional): Whether to load and continue training an existing model (True) or create a new model (False). Defaults to False. Note: - For 'Residual' models, the number of inputs is doubled during fitting - Learning rate scheduling with exponential decay is applied for existing models - The model is automatically saved after training completion """ #if model_settings is not None: self.model_settings = model_settings # else: # self.model_settings = Model_Settings() self.input_dim = len(training_features[0]) self.output_dim = np.shape(training_targets)[1] # self.validation_features = validation_features.astype(np.float16) # self.validation_targets = validation_targets.astype(np.float16) if existing == False: match model_type: case 'Residual': print('Residual') self.residual_model(self.model_settings.inputs) self.fitting(training_features, training_targets, validation_features, validation_targets, self.model_settings.inputs*2) case _: print('Sequential') self.model = tf.keras.models.Sequential() self.setup_model() self.fitting(training_features, training_targets, validation_features, validation_targets, self.model_settings.inputs) self.saveing(dir) else: lr_schedule = keras.optimizers.schedules.ExponentialDecay( initial_learning_rate=self.model_settings.inital_learning_rate, decay_steps=1e6, decay_rate=self.model_settings.decay_rate, staircase=True ) self.dir = dir match model_type: case 'Residual': print('Residual') #self.residual_model_existing(self.model_settings.inputs) self.model = self.load_existing_model(dir) model_optimiser = tf.keras.optimizers.Adam(learning_rate=lr_schedule) self.model.compile(optimizer=model_optimiser, loss=self.model_settings.loss_function, metrics=self.model_settings.metrics) self.fitting(training_features, training_targets, validation_features, validation_targets, self.model_settings.inputs*2) case _: print('Sequential') #self.model = tf.keras.models.Sequential() #self.setup_model_existing() self.model = self.load_existing_model(dir) model_optimiser = tf.keras.optimizers.Adam(learning_rate=lr_schedule) self.model.compile(optimizer=model_optimiser, loss=self.model_settings.loss_function, metrics=self.model_settings.metrics) self.fitting(training_features, training_targets, validation_features, validation_targets, self.model_settings.inputs) self.saveing_existing(dir) def load_existing_model(self, dir: str) -> tf.keras.Model: """ Load an existing trained model from the specified directory. This method loads a previously saved Keras model, allowing for continued training or inference. It handles the model loading process and returns the loaded model instance. Args: dir (str): Directory path where the model files are stored. The model should be saved in Keras format (.keras file). Returns: tf.keras.Model: The loaded Keras model ready for training or inference. Raises: FileNotFoundError: If the model file is not found in the specified directory. ValueError: If the model file is corrupted or incompatible. """ model_path = os.path.join(dir, 'model.keras') if os.path.exists(model_path): m = keras.models.load_model(model_path, compile=False) #m.save_weights(os.path.join(dir, 'model_weights.weights.h5')) return m else: raise FileNotFoundError(f"Model file not found at {model_path}") def setup_model(self) -> None: """ Set up a standard sequential model based on the provided model settings. This method adds layers to the model, including dense layers, activation functions, batch normalization, and dropout layers. """ for i, nodes in enumerate(self.model_settings.layer_nodes): if i == 0: self.model.add(tf.keras.layers.Dense(nodes, kernel_initializer=self.model_settings.initializer, kernel_regularizer=self.model_settings.regularization)) self.model.add(tf.keras.layers.Activation(self.model_settings.activation)) if self.model_settings.batch_norm: self.model.add(tf.keras.layers.BatchNormalization()) if len(self.model_settings.dropout) != 0: self.model.add(tf.keras.layers.Dropout(self.model_settings.dropout[i])) else: self.model.add(tf.keras.layers.Dense(nodes, kernel_initializer=self.model_settings.initializer, kernel_regularizer=self.model_settings.regularization)) self.model.add(tf.keras.layers.Activation(self.model_settings.activation)) if self.model_settings.batch_norm: self.model.add(tf.keras.layers.BatchNormalization()) if len(self.model_settings.dropout) != 0: self.model.add(tf.keras.layers.Dropout(self.model_settings.dropout[i])) self.model.add(tf.keras.layers.Dense(self.output_dim, activation='tanh')) lr_schedule = keras.optimizers.schedules.ExponentialDecay( initial_learning_rate=self.model_settings.inital_learning_rate, decay_steps=1e6, decay_rate=self.model_settings.decay_rate, staircase=True ) model_optimiser = tf.keras.optimizers.Adam(learning_rate=lr_schedule) self.model.compile(optimizer=model_optimiser, loss=self.model_settings.loss_function, metrics=self.model_settings.metrics) def setup_model_existing(self) -> None: """ Set up a standard sequential model based on the provided model settings. This method adds layers to the model, including dense layers, activation functions, batch normalization, and dropout layers. """ for i, nodes in enumerate(self.model_settings.layer_nodes): if i == 0: self.model.add(tf.keras.layers.Dense(nodes, kernel_initializer=self.model_settings.initializer, kernel_regularizer=self.model_settings.regularization)) self.model.add(tf.keras.layers.Activation(self.model_settings.activation)) if self.model_settings.batch_norm: self.model.add(tf.keras.layers.BatchNormalization()) if len(self.model_settings.dropout) != 0: self.model.add(tf.keras.layers.Dropout(self.model_settings.dropout[i])) else: self.model.add(tf.keras.layers.Dense(nodes, kernel_initializer=self.model_settings.initializer, kernel_regularizer=self.model_settings.regularization)) self.model.add(tf.keras.layers.Activation(self.model_settings.activation)) if self.model_settings.batch_norm: self.model.add(tf.keras.layers.BatchNormalization()) if len(self.model_settings.dropout) != 0: self.model.add(tf.keras.layers.Dropout(self.model_settings.dropout[i])) self.model.add(tf.keras.layers.Dense(self.output_dim, activation='tanh')) lr_schedule = keras.optimizers.schedules.ExponentialDecay( initial_learning_rate=self.model_settings.inital_learning_rate, decay_steps=1e6, decay_rate=self.model_settings.decay_rate, staircase=True ) model_optimiser = tf.keras.optimizers.Adam(learning_rate=lr_schedule) self.model.compile(optimizer=model_optimiser, loss=self.model_settings.loss_function, metrics=self.model_settings.metrics) self.model.load_weights(os.path.join(self.dir, 'model_weights.weights.h5')) def residual_block(self, inputs: tf.Tensor, n_layers: int, nodes: int, activation: str, dropout: Optional[float] = None) -> tf.Tensor: """ Create a residual block for the model. Args: inputs (tensorflow.Tensor): Input tensor for the block. n_layers (int): Number of layers in the block. nodes (int): Number of nodes in each layer. activation (str): Activation function to use. dropout (float, optional): Dropout rate. Returns: tensorflow.Tensor: Output tensor of the residual block. """ if self.model_settings.batch_norm: x = layers.BatchNormalization()(inputs) x = layers.Dense(nodes, activation=activation)(x) for idx in range(n_layers): x = layers.Dense(nodes, activation=activation)(x) if dropout is not None: x = layers.Dropout(dropout)(x) x = layers.Add()([inputs, x]) return x def setup_residual_model(self) -> None: """ Set up a residual model with a single input. This method creates a model with multiple residual blocks and compiles it. """ inputs = keras.Input(shape=(self.input_dim,)) x = layers.Dense(self.model_settings.layer_nodes[0], activation=None)(inputs) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) outputs = layers.Dense(self.output_dim, activation=None)(x) self.model = keras.Model(inputs=inputs, outputs=outputs) model_optimiser = tf.keras.optimizers.Adam(learning_rate=lr(self.model_settings.inital_learning_rate, self.model_settings.gamma_learning_rate, self.model_settings.power_learning_rate)) self.model.compile(optimizer=model_optimiser, loss=self.model_settings.loss_function, metrics=self.model_settings.metrics) def setup_2input_residual_model(self) -> None: """ Set up a residual model with two inputs. This method creates a model with two input branches, each with residual blocks, and combines them using subtraction. """ input_1 = keras.Input(shape=(int(self.input_dim/2),)) input_2 = keras.Input(shape=(int(self.input_dim/2),)) x_1 = layers.Dense(self.model_settings.layer_nodes[0], activation=None)(input_1) x_2 = layers.Dense(self.model_settings.layer_nodes[0], activation=None)(input_2) x_1 = self.residual_block(x_1, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_1 = self.residual_block(x_1, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_1 = self.residual_block(x_1, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_1 = self.residual_block(x_1, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_1 = self.residual_block(x_1, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_1 = self.residual_block(x_1, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_1 = self.residual_block(x_1, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_1 = self.residual_block(x_1, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_2 = self.residual_block(x_2, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_2 = self.residual_block(x_2, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_2 = self.residual_block(x_2, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_2 = self.residual_block(x_2, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_2 = self.residual_block(x_2, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_2 = self.residual_block(x_2, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_2 = self.residual_block(x_2, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_2 = self.residual_block(x_2, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = layers.Subtract()([x_1, x_2]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) outputs = layers.Dense(self.output_dim, activation=None)(x) self.model = keras.Model(inputs=[input_1,input_2], outputs=outputs) model_optimiser = tf.keras.optimizers.Adam(learning_rate=lr(self.model_settings.inital_learning_rate, self.model_settings.gamma_learning_rate, self.model_settings.power_learning_rate)) self.model.compile(optimizer=model_optimiser, loss=self.model_settings.loss_function, metrics=self.model_settings.metrics) def residual_model(self, inputs: int =1) -> None: print('inputs: ', inputs) match inputs*2: case 1: self.residual_model_1_input() case 2: self.residual_model_2_input() case 4: self.residual_model_4_input() case 8: self.residual_model_8_input() case _: self.residual_model_1_input() def residual_model_existing(self, inputs: int =1) -> None: print('inputs: ', inputs) match inputs*2: case 1: self.residual_model_1_input() case 2: self.residual_model_2_input_existing() case 4: self.residual_model_4_input() case 8: self.residual_model_8_input() case _: self.residual_model_1_input() def residual_model_1_input(self) -> None: """ Set up a residual model with a single input. This method creates a model with multiple residual blocks and compiles it. """ inputs = keras.Input(shape=(self.input_dim,)) x = layers.Dense(self.model_settings.layer_nodes[0], activation=None)(inputs) for i in range(len(self.model_settings.layer_nodes)): x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) outputs = layers.Dense(self.output_dim, activation='tanh')(x) self.model = keras.Model(inputs=inputs, outputs=outputs) model_optimiser = tf.keras.optimizers.Adam(learning_rate=lr(self.model_settings.inital_learning_rate, self.model_settings.gamma_learning_rate, self.model_settings.power_learning_rate)) self.model.compile(optimizer=model_optimiser, loss=self.model_settings.loss_function, metrics=self.model_settings.metrics) def residual_model_2_input(self) -> None: """ Set up a residual model with two inputs. This method creates a model with two input branches, each with residual blocks, and combines them using subtraction. """ input_1 = keras.Input(shape=(int(self.input_dim/2),),name='input_1') input_2 = keras.Input(shape=(int(self.input_dim/2),),name='input_2') x_1 = layers.Dense(self.model_settings.layer_nodes[0], activation=None)(input_1)# x_2 = layers.Dense(self.model_settings.layer_nodes[0], activation=None)(input_2) for i in range(len(self.model_settings.layer_nodes)): x_1 = self.residual_block(x_1, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_2 = self.residual_block(x_2, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = layers.Subtract()([x_1, x_2]) for i in range(len(self.model_settings.layer_nodes)): x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) #O_A = keras.actiavation.tanh() outputs = layers.Dense(self.output_dim, activation='tanh')(x) self.model = keras.Model(inputs=(input_1,input_2), outputs=outputs) lr_schedule = keras.optimizers.schedules.ExponentialDecay( initial_learning_rate=self.model_settings.inital_learning_rate, decay_steps=1e6, decay_rate=self.model_settings.decay_rate, staircase=True ) model_optimiser = tf.keras.optimizers.Adam(learning_rate=lr_schedule) self.model.compile(optimizer=model_optimiser, loss=self.model_settings.loss_function, metrics=self.model_settings.metrics) def residual_model_2_input_existing(self) -> None: """ Set up a residual model with two inputs. This method creates a model with two input branches, each with residual blocks, and combines them using subtraction. """ input_1 = keras.Input(shape=(int(self.input_dim/2),),name='input_1') input_2 = keras.Input(shape=(int(self.input_dim/2),),name='input_2') x_1 = layers.Dense(self.model_settings.layer_nodes[0], activation=None)(input_1)# x_2 = layers.Dense(self.model_settings.layer_nodes[0], activation=None)(input_2) for i in range(len(self.model_settings.layer_nodes)): x_1 = self.residual_block(x_1, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_2 = self.residual_block(x_2, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = layers.Subtract()([x_1, x_2]) for i in range(len(self.model_settings.layer_nodes)): x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) #O_A = keras.actiavation.tanh() outputs = layers.Dense(self.output_dim, activation='tanh')(x) self.model = keras.Model(inputs=(input_1,input_2), outputs=outputs) lr_schedule = keras.optimizers.schedules.ExponentialDecay( initial_learning_rate=self.model_settings.inital_learning_rate, decay_steps=1e6, decay_rate=self.model_settings.decay_rate, staircase=True ) model_optimiser = tf.keras.optimizers.Adam(learning_rate=lr_schedule) self.model.compile(optimizer=model_optimiser, loss=self.model_settings.loss_function, metrics=self.model_settings.metrics) self.model.load_weights(os.path.join(self.dir, 'model_weights.weights.h5')) def residual_model_4_input(self) -> None: """ Set up a residual model with four inputs. This method creates a model with four input branches, each with residual blocks, and combines them using subtraction. """ input_1 = keras.Input(shape=(int(self.input_dim/4),)) input_2 = keras.Input(shape=(int(self.input_dim/4),)) input_3 = keras.Input(shape=(int(self.input_dim/4),)) input_4 = keras.Input(shape=(int(self.input_dim/4),)) x_1 = layers.Dense(self.model_settings.layer_nodes[0], activation=None)(input_1) x_2 = layers.Dense(self.model_settings.layer_nodes[0], activation=None)(input_2) x_3 = layers.Dense(self.model_settings.layer_nodes[0], activation=None)(input_3) x_4 = layers.Dense(self.model_settings.layer_nodes[0], activation=None)(input_4) for i in range(len(self.model_settings.layer_nodes)): x_1 = self.residual_block(x_1, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_2 = self.residual_block(x_2, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_3 = self.residual_block(x_3, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_4 = self.residual_block(x_4, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_1 = layers.Subtract()([x_1, x_2]) x_3 = layers.Subtract()([x_3, x_4]) for i in range(len(self.model_settings.layer_nodes)): x_1 = self.residual_block(x_1, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_3 = self.residual_block(x_3, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = layers.Subtract()([x_1, x_3]) for i in range(len(self.model_settings.layer_nodes)): x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) outputs = layers.Dense(self.output_dim, activation=None)(x) self.model = keras.Model(inputs=[input_1,input_2,input_3,input_4], outputs=outputs) model_optimiser = tf.keras.optimizers.Adam(learning_rate=lr(self.model_settings.inital_learning_rate, self.model_settings.gamma_learning_rate, self.model_settings.power_learning_rate)) self.model.compile(optimizer=model_optimiser, loss=self.model_settings.loss_function, metrics=self.model_settings.metrics) def residual_model_8_input(self) -> None: """ Set up a residual model with eight inputs. This method creates a model with eight input branches, each with residual blocks, and combines them using subtraction. """ input_1 = keras.Input(shape=(int(self.input_dim/8),)) input_2 = keras.Input(shape=(int(self.input_dim/8),)) input_3 = keras.Input(shape=(int(self.input_dim/8),)) input_4 = keras.Input(shape=(int(self.input_dim/8),)) input_5 = keras.Input(shape=(int(self.input_dim/8),)) input_6 = keras.Input(shape=(int(self.input_dim/8),)) input_7 = keras.Input(shape=(int(self.input_dim/8),)) input_8 = keras.Input(shape=(int(self.input_dim/8),)) x_1 = layers.Dense(self.model_settings.layer_nodes[0], activation=None)(input_1) x_2 = layers.Dense(self.model_settings.layer_nodes[0], activation=None)(input_2) x_3 = layers.Dense(self.model_settings.layer_nodes[0], activation=None)(input_3) x_4 = layers.Dense(self.model_settings.layer_nodes[0], activation=None)(input_4) x_5 = layers.Dense(self.model_settings.layer_nodes[0], activation=None)(input_5) x_6 = layers.Dense(self.model_settings.layer_nodes[0], activation=None)(input_6) x_7 = layers.Dense(self.model_settings.layer_nodes[0], activation=None)(input_7) x_8 = layers.Dense(self.model_settings.layer_nodes[0], activation=None)(input_8) for i in range(len(self.model_settings.layer_nodes)): x_1 = self.residual_block(x_1, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_2 = self.residual_block(x_2, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_3 = self.residual_block(x_3, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_4 = self.residual_block(x_4, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_5 = self.residual_block(x_5, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_6 = self.residual_block(x_6, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_7 = self.residual_block(x_7, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_8 = self.residual_block(x_8, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_1 = layers.Subtract()([x_1, x_2]) x_3 = layers.Subtract()([x_3, x_4]) x_5 = layers.Subtract()([x_5, x_6]) x_7 = layers.Subtract()([x_7, x_8]) for i in range(len(self.model_settings.layer_nodes)): x_1 = self.residual_block(x_1, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_3 = self.residual_block(x_3, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_5 = self.residual_block(x_5, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_7 = self.residual_block(x_7, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_1 = layers.Subtract()([x_1, x_3]) x_5 = layers.Subtract()([x_5, x_7]) for i in range(len(self.model_settings.layer_nodes)): x_1 = self.residual_block(x_1, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_5 = self.residual_block(x_5, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = layers.Subtract()([x_1, x_5]) for i in range(len(self.model_settings.layer_nodes)): x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) outputs = layers.Dense(self.output_dim, activation=None)(x) self.model = keras.Model(inputs=[input_1,input_2,input_3,input_4,input_5,input_6,input_7,input_8], outputs=outputs) model_optimiser = tf.keras.optimizers.Adam(learning_rate=lr(self.model_settings.inital_learning_rate, self.model_settings.gamma_learning_rate, self.model_settings.power_learning_rate)) self.model.compile(optimizer=model_optimiser, loss=self.model_settings.loss_function, metrics=self.model_settings.metrics) def fitting(self, x: np.ndarray, y: np.ndarray, val_x: np.ndarray, val_y: np.ndarray, inputs: int) -> float: print('fitting:', inputs) match inputs: case 1: self.fitting_1_input(x, y, val_x, val_y) case 2: self.fitting_2_input(x, y, val_x, val_y) case 4: self.fitting_4_input(x, y, val_x, val_y) case 8: self.fitting_8_input(x, y, val_x, val_y) case _: self.fitting_1_input(x, y, val_x, val_y) def fitting_1_input(self, x: np.ndarray, y: np.ndarray, val_x: np.ndarray, val_y: np.ndarray) -> float: """ Train the model using the provided training and validation data. Args: x (numpy.ndarray): Training features. y (numpy.ndarray): Training targets. val_x (numpy.ndarray): Validation features. val_y (numpy.ndarray): Validation targets. Returns: float: The final validation mean absolute error. """ callbacks = [tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=self.model_settings.patience, verbose=1)] history = self.model.fit( x=x, y=y, validation_data=(val_x, val_y), epochs=self.model_settings.epochs, callbacks=callbacks, shuffle=True, batch_size=int(self.model_settings.batch_size), ) return history.history['val_mean_absolute_error'][-1] # """ # Train the model using the provided training and validation data. # Args: # x (numpy.ndarray): Training features. # y (numpy.ndarray): Training targets. # val_x (numpy.ndarray): Validation features. # val_y (numpy.ndarray): Validation targets. # Returns: # float: The final validation mean absolute error. # """ # print('fitting_2_input') # callbacks = [tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=self.model_settings.patience, verbose=1)] # history = self.model.fit( # x=(x[:, :int(len(x[0])/2)], x[:, int(len(x[0])/2):]), # y=y, # validation_data=([ val_x[:, :int(len(x[0])/2)], val_x[:, int(len(x[0])/2):]], val_y), # epochs=self.model_settings.epochs, # callbacks=callbacks, # shuffle=True, # batch_size=int(self.model_settings.batch_size), # ) # return history.history['val_mean_absolute_error'][-1] def fitting_2_input(self, x: np.ndarray, y: np.ndarray, val_x: np.ndarray, val_y: np.ndarray) -> float: """ Train the model using the provided training and validation data. Args: x (numpy.ndarray): Training features. y (numpy.ndarray): Training targets. val_x (numpy.ndarray): Validation features. val_y (numpy.ndarray): Validation targets. Returns: float: The final validation mean absolute error. """ print('fitting_2_input') callbacks = [tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=self.model_settings.patience, verbose=1)] history = self.model.fit( x=(x[:, :int(len(x[0])/2)], x[:, int(len(x[0])/2):]), y=y, validation_data=([ val_x[:, :int(len(x[0])/2)], val_x[:, int(len(x[0])/2):]], val_y), epochs=self.model_settings.epochs, callbacks=callbacks, shuffle=True, batch_size=int(self.model_settings.batch_size), ) return history.history['val_mean_absolute_error'][-1] def fitting_4_input(self, x: np.ndarray, y: np.ndarray, val_x: np.ndarray, val_y: np.ndarray) -> float: """ Train the model using the provided training and validation data. Args: x (numpy.ndarray): Training features. y (numpy.ndarray): Training targets. val_x (numpy.ndarray): Validation features. val_y (numpy.ndarray): Validation targets. Returns: float: The final validation mean absolute error. """ callbacks = [tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=self.model_settings.patience, verbose=1)] history = self.model.fit( x=[x[:, :int(len(x[0])*1/4)], x[:, int(len(x[0])*2/4):int(len(x[0])*3/4)], x[:, int(len(x[0])*1/4):int(len(x[0])*2/4)], x[:, int(len(x[0])*3/4):]], y=y, validation_data=([ val_x[:, :int(len(x[0])*1/4)], val_x[:, int(len(x[0])*2/4):int(len(x[0])*3/4)], val_x[:, int(len(x[0])*1/4):int(len(x[0])*2/4)], val_x[:, int(len(x[0])*3/4):]], val_y), epochs=self.model_settings.epochs, callbacks=callbacks, shuffle=True, batch_size=int(self.model_settings.batch_size), ) return history.history['val_mean_absolute_error'][-1] def fitting_8_input(self, x: np.ndarray, y: np.ndarray, val_x: np.ndarray, val_y: np.ndarray) -> float: """ Train the model using the provided training and validation data. Args: x (numpy.ndarray): Training features. y (numpy.ndarray): Training targets. val_x (numpy.ndarray): Validation features. val_y (numpy.ndarray): Validation targets. Returns: float: The final validation mean absolute error. """ callbacks = [tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=self.model_settings.patience, verbose=1)] print(np.shape(x)) print(np.shape(val_x)) print(np.shape(x[0])) print(np.shape(val_x[0])) history = self.model.fit( x=[x[:, :int(len(x[0]) * 1/8)], x[:, int(len(x[0]) * 4/8):int(len(x[0])* 5/8)], x[:, int(len(x[0]) * 1/8):int(len(x[0])* 2/8)], x[:, int(len(x[0])* 5/8):int(len(x[0])* 6/8)], x[:, int(len(x[0])* 2/8):int(len(x[0])* 3/8)], x[:, int(len(x[0])*6/8):int(len(x[0])* 7/8)], x[:, int(len(x[0])*3/8):int(len(x[0])*4/8)], x[:, int(len(x[0])*7/8):]], y=y, validation_data=([val_x[:, :int(len(x[0])* 1/8)], val_x[:, int(len(x[0]) * 4/8):int(len(x[0]) * 5/8)], val_x[:, int(len(x[0])*1/8):int(len(x[0])*2/8)], val_x[:, int(len(x[0])*5/8):int(len(x[0])*6/8)], val_x[:, int(len(x[0])*2/8):int(len(x[0])*3/8)], val_x[:, int(len(x[0])*6/8):int(len(x[0])*7/8)], val_x[:, int(len(x[0])*3/8):int(len(x[0])*4/8)], val_x[:, int(len(x[0])*7/8):]], val_y), epochs=self.model_settings.epochs, callbacks=callbacks, shuffle=True, batch_size=int(self.model_settings.batch_size), ) return history.history['val_mean_absolute_error'][-1] def saveing(self, dir: str) -> None: """ Save the trained model to the specified directory. Args: dir (str): Directory to save the model. """ dir = os.path.join(dir, 'model.keras') self.model.save(dir) def saveing_existing(self, dir: str) -> None: """ Save the trained model to the specified directory. Args: dir (str): Directory to save the model. """ files = str(len(os.listdir(dir)) + 1) print('Saving model to:', dir) dir = os.path.join(dir, 'model'+files+'.keras') self.model.save(dir)
A class for training machine learning models using TensorFlow/Keras.
This class supports different model architectures, including residual models, and provides methods for training, saving, and configuring models.
Initialize a Training instance and train the model.
This method sets up and trains a neural network model using the provided training and validation data. It supports both creating new models and continuing training from existing models.
Args
training_features
:numpy.ndarray
- Input features for training the model.
training_targets
:numpy.ndarray
- Target values for training the model.
validation_features
:numpy.ndarray
- Input features for model validation.
validation_targets
:numpy.ndarray
- Target values for model validation.
dir
:str
- Directory path where the trained model will be saved.
model_settings
:Model_Settings
, optional- Configuration settings for the model including architecture, hyperparameters, and training parameters.
model_type
:str
, optional- Type of model architecture to use. Supported types include 'Residual' and default Sequential models.
existing
:bool
, optional- Whether to load and continue training an existing model (True) or create a new model (False). Defaults to False.
Note
- For 'Residual' models, the number of inputs is doubled during fitting
- Learning rate scheduling with exponential decay is applied for existing models
- The model is automatically saved after training completion
Methods
def fitting(self,
x: numpy.ndarray,
y: numpy.ndarray,
val_x: numpy.ndarray,
val_y: numpy.ndarray,
inputs: int) ‑> float-
Expand source code
def fitting(self, x: np.ndarray, y: np.ndarray, val_x: np.ndarray, val_y: np.ndarray, inputs: int) -> float: print('fitting:', inputs) match inputs: case 1: self.fitting_1_input(x, y, val_x, val_y) case 2: self.fitting_2_input(x, y, val_x, val_y) case 4: self.fitting_4_input(x, y, val_x, val_y) case 8: self.fitting_8_input(x, y, val_x, val_y) case _: self.fitting_1_input(x, y, val_x, val_y)
def fitting_1_input(self,
x: numpy.ndarray,
y: numpy.ndarray,
val_x: numpy.ndarray,
val_y: numpy.ndarray) ‑> float-
Expand source code
def fitting_1_input(self, x: np.ndarray, y: np.ndarray, val_x: np.ndarray, val_y: np.ndarray) -> float: """ Train the model using the provided training and validation data. Args: x (numpy.ndarray): Training features. y (numpy.ndarray): Training targets. val_x (numpy.ndarray): Validation features. val_y (numpy.ndarray): Validation targets. Returns: float: The final validation mean absolute error. """ callbacks = [tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=self.model_settings.patience, verbose=1)] history = self.model.fit( x=x, y=y, validation_data=(val_x, val_y), epochs=self.model_settings.epochs, callbacks=callbacks, shuffle=True, batch_size=int(self.model_settings.batch_size), ) return history.history['val_mean_absolute_error'][-1] # """ # Train the model using the provided training and validation data. # Args: # x (numpy.ndarray): Training features. # y (numpy.ndarray): Training targets. # val_x (numpy.ndarray): Validation features. # val_y (numpy.ndarray): Validation targets. # Returns: # float: The final validation mean absolute error. # """ # print('fitting_2_input') # callbacks = [tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=self.model_settings.patience, verbose=1)] # history = self.model.fit( # x=(x[:, :int(len(x[0])/2)], x[:, int(len(x[0])/2):]), # y=y, # validation_data=([ val_x[:, :int(len(x[0])/2)], val_x[:, int(len(x[0])/2):]], val_y), # epochs=self.model_settings.epochs, # callbacks=callbacks, # shuffle=True, # batch_size=int(self.model_settings.batch_size), # ) # return history.history['val_mean_absolute_error'][-1]
Train the model using the provided training and validation data.
Args
x
:numpy.ndarray
- Training features.
y
:numpy.ndarray
- Training targets.
val_x
:numpy.ndarray
- Validation features.
val_y
:numpy.ndarray
- Validation targets.
Returns
float
- The final validation mean absolute error.
def fitting_2_input(self,
x: numpy.ndarray,
y: numpy.ndarray,
val_x: numpy.ndarray,
val_y: numpy.ndarray) ‑> float-
Expand source code
def fitting_2_input(self, x: np.ndarray, y: np.ndarray, val_x: np.ndarray, val_y: np.ndarray) -> float: """ Train the model using the provided training and validation data. Args: x (numpy.ndarray): Training features. y (numpy.ndarray): Training targets. val_x (numpy.ndarray): Validation features. val_y (numpy.ndarray): Validation targets. Returns: float: The final validation mean absolute error. """ print('fitting_2_input') callbacks = [tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=self.model_settings.patience, verbose=1)] history = self.model.fit( x=(x[:, :int(len(x[0])/2)], x[:, int(len(x[0])/2):]), y=y, validation_data=([ val_x[:, :int(len(x[0])/2)], val_x[:, int(len(x[0])/2):]], val_y), epochs=self.model_settings.epochs, callbacks=callbacks, shuffle=True, batch_size=int(self.model_settings.batch_size), ) return history.history['val_mean_absolute_error'][-1]
Train the model using the provided training and validation data.
Args
x
:numpy.ndarray
- Training features.
y
:numpy.ndarray
- Training targets.
val_x
:numpy.ndarray
- Validation features.
val_y
:numpy.ndarray
- Validation targets.
Returns
float
- The final validation mean absolute error.
def fitting_4_input(self,
x: numpy.ndarray,
y: numpy.ndarray,
val_x: numpy.ndarray,
val_y: numpy.ndarray) ‑> float-
Expand source code
def fitting_4_input(self, x: np.ndarray, y: np.ndarray, val_x: np.ndarray, val_y: np.ndarray) -> float: """ Train the model using the provided training and validation data. Args: x (numpy.ndarray): Training features. y (numpy.ndarray): Training targets. val_x (numpy.ndarray): Validation features. val_y (numpy.ndarray): Validation targets. Returns: float: The final validation mean absolute error. """ callbacks = [tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=self.model_settings.patience, verbose=1)] history = self.model.fit( x=[x[:, :int(len(x[0])*1/4)], x[:, int(len(x[0])*2/4):int(len(x[0])*3/4)], x[:, int(len(x[0])*1/4):int(len(x[0])*2/4)], x[:, int(len(x[0])*3/4):]], y=y, validation_data=([ val_x[:, :int(len(x[0])*1/4)], val_x[:, int(len(x[0])*2/4):int(len(x[0])*3/4)], val_x[:, int(len(x[0])*1/4):int(len(x[0])*2/4)], val_x[:, int(len(x[0])*3/4):]], val_y), epochs=self.model_settings.epochs, callbacks=callbacks, shuffle=True, batch_size=int(self.model_settings.batch_size), ) return history.history['val_mean_absolute_error'][-1]
Train the model using the provided training and validation data.
Args
x
:numpy.ndarray
- Training features.
y
:numpy.ndarray
- Training targets.
val_x
:numpy.ndarray
- Validation features.
val_y
:numpy.ndarray
- Validation targets.
Returns
float
- The final validation mean absolute error.
def fitting_8_input(self,
x: numpy.ndarray,
y: numpy.ndarray,
val_x: numpy.ndarray,
val_y: numpy.ndarray) ‑> float-
Expand source code
def fitting_8_input(self, x: np.ndarray, y: np.ndarray, val_x: np.ndarray, val_y: np.ndarray) -> float: """ Train the model using the provided training and validation data. Args: x (numpy.ndarray): Training features. y (numpy.ndarray): Training targets. val_x (numpy.ndarray): Validation features. val_y (numpy.ndarray): Validation targets. Returns: float: The final validation mean absolute error. """ callbacks = [tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=self.model_settings.patience, verbose=1)] print(np.shape(x)) print(np.shape(val_x)) print(np.shape(x[0])) print(np.shape(val_x[0])) history = self.model.fit( x=[x[:, :int(len(x[0]) * 1/8)], x[:, int(len(x[0]) * 4/8):int(len(x[0])* 5/8)], x[:, int(len(x[0]) * 1/8):int(len(x[0])* 2/8)], x[:, int(len(x[0])* 5/8):int(len(x[0])* 6/8)], x[:, int(len(x[0])* 2/8):int(len(x[0])* 3/8)], x[:, int(len(x[0])*6/8):int(len(x[0])* 7/8)], x[:, int(len(x[0])*3/8):int(len(x[0])*4/8)], x[:, int(len(x[0])*7/8):]], y=y, validation_data=([val_x[:, :int(len(x[0])* 1/8)], val_x[:, int(len(x[0]) * 4/8):int(len(x[0]) * 5/8)], val_x[:, int(len(x[0])*1/8):int(len(x[0])*2/8)], val_x[:, int(len(x[0])*5/8):int(len(x[0])*6/8)], val_x[:, int(len(x[0])*2/8):int(len(x[0])*3/8)], val_x[:, int(len(x[0])*6/8):int(len(x[0])*7/8)], val_x[:, int(len(x[0])*3/8):int(len(x[0])*4/8)], val_x[:, int(len(x[0])*7/8):]], val_y), epochs=self.model_settings.epochs, callbacks=callbacks, shuffle=True, batch_size=int(self.model_settings.batch_size), ) return history.history['val_mean_absolute_error'][-1]
Train the model using the provided training and validation data.
Args
x
:numpy.ndarray
- Training features.
y
:numpy.ndarray
- Training targets.
val_x
:numpy.ndarray
- Validation features.
val_y
:numpy.ndarray
- Validation targets.
Returns
float
- The final validation mean absolute error.
def load_existing_model(self, dir: str) ‑> keras.src.models.model.Model
-
Expand source code
def load_existing_model(self, dir: str) -> tf.keras.Model: """ Load an existing trained model from the specified directory. This method loads a previously saved Keras model, allowing for continued training or inference. It handles the model loading process and returns the loaded model instance. Args: dir (str): Directory path where the model files are stored. The model should be saved in Keras format (.keras file). Returns: tf.keras.Model: The loaded Keras model ready for training or inference. Raises: FileNotFoundError: If the model file is not found in the specified directory. ValueError: If the model file is corrupted or incompatible. """ model_path = os.path.join(dir, 'model.keras') if os.path.exists(model_path): m = keras.models.load_model(model_path, compile=False) #m.save_weights(os.path.join(dir, 'model_weights.weights.h5')) return m else: raise FileNotFoundError(f"Model file not found at {model_path}")
Load an existing trained model from the specified directory.
This method loads a previously saved Keras model, allowing for continued training or inference. It handles the model loading process and returns the loaded model instance.
Args
dir
:str
- Directory path where the model files are stored. The model should be saved in Keras format (.keras file).
Returns
tf.keras.Model
- The loaded Keras model ready for training or inference.
Raises
FileNotFoundError
- If the model file is not found in the specified directory.
ValueError
- If the model file is corrupted or incompatible.
def residual_block(self,
inputs: tensorflow.python.framework.tensor.Tensor,
n_layers: int,
nodes: int,
activation: str,
dropout: float | None = None) ‑> tensorflow.python.framework.tensor.Tensor-
Expand source code
def residual_block(self, inputs: tf.Tensor, n_layers: int, nodes: int, activation: str, dropout: Optional[float] = None) -> tf.Tensor: """ Create a residual block for the model. Args: inputs (tensorflow.Tensor): Input tensor for the block. n_layers (int): Number of layers in the block. nodes (int): Number of nodes in each layer. activation (str): Activation function to use. dropout (float, optional): Dropout rate. Returns: tensorflow.Tensor: Output tensor of the residual block. """ if self.model_settings.batch_norm: x = layers.BatchNormalization()(inputs) x = layers.Dense(nodes, activation=activation)(x) for idx in range(n_layers): x = layers.Dense(nodes, activation=activation)(x) if dropout is not None: x = layers.Dropout(dropout)(x) x = layers.Add()([inputs, x]) return x
Create a residual block for the model.
Args
inputs
:tensorflow.Tensor
- Input tensor for the block.
n_layers
:int
- Number of layers in the block.
nodes
:int
- Number of nodes in each layer.
activation
:str
- Activation function to use.
dropout
:float
, optional- Dropout rate.
Returns
tensorflow.Tensor
- Output tensor of the residual block.
def residual_model(self, inputs: int = 1) ‑> None
-
Expand source code
def residual_model(self, inputs: int =1) -> None: print('inputs: ', inputs) match inputs*2: case 1: self.residual_model_1_input() case 2: self.residual_model_2_input() case 4: self.residual_model_4_input() case 8: self.residual_model_8_input() case _: self.residual_model_1_input()
def residual_model_1_input(self) ‑> None
-
Expand source code
def residual_model_1_input(self) -> None: """ Set up a residual model with a single input. This method creates a model with multiple residual blocks and compiles it. """ inputs = keras.Input(shape=(self.input_dim,)) x = layers.Dense(self.model_settings.layer_nodes[0], activation=None)(inputs) for i in range(len(self.model_settings.layer_nodes)): x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) outputs = layers.Dense(self.output_dim, activation='tanh')(x) self.model = keras.Model(inputs=inputs, outputs=outputs) model_optimiser = tf.keras.optimizers.Adam(learning_rate=lr(self.model_settings.inital_learning_rate, self.model_settings.gamma_learning_rate, self.model_settings.power_learning_rate)) self.model.compile(optimizer=model_optimiser, loss=self.model_settings.loss_function, metrics=self.model_settings.metrics)
Set up a residual model with a single input.
This method creates a model with multiple residual blocks and compiles it.
def residual_model_2_input(self) ‑> None
-
Expand source code
def residual_model_2_input(self) -> None: """ Set up a residual model with two inputs. This method creates a model with two input branches, each with residual blocks, and combines them using subtraction. """ input_1 = keras.Input(shape=(int(self.input_dim/2),),name='input_1') input_2 = keras.Input(shape=(int(self.input_dim/2),),name='input_2') x_1 = layers.Dense(self.model_settings.layer_nodes[0], activation=None)(input_1)# x_2 = layers.Dense(self.model_settings.layer_nodes[0], activation=None)(input_2) for i in range(len(self.model_settings.layer_nodes)): x_1 = self.residual_block(x_1, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_2 = self.residual_block(x_2, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = layers.Subtract()([x_1, x_2]) for i in range(len(self.model_settings.layer_nodes)): x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) #O_A = keras.actiavation.tanh() outputs = layers.Dense(self.output_dim, activation='tanh')(x) self.model = keras.Model(inputs=(input_1,input_2), outputs=outputs) lr_schedule = keras.optimizers.schedules.ExponentialDecay( initial_learning_rate=self.model_settings.inital_learning_rate, decay_steps=1e6, decay_rate=self.model_settings.decay_rate, staircase=True ) model_optimiser = tf.keras.optimizers.Adam(learning_rate=lr_schedule) self.model.compile(optimizer=model_optimiser, loss=self.model_settings.loss_function, metrics=self.model_settings.metrics)
Set up a residual model with two inputs.
This method creates a model with two input branches, each with residual blocks, and combines them using subtraction.
def residual_model_2_input_existing(self) ‑> None
-
Expand source code
def residual_model_2_input_existing(self) -> None: """ Set up a residual model with two inputs. This method creates a model with two input branches, each with residual blocks, and combines them using subtraction. """ input_1 = keras.Input(shape=(int(self.input_dim/2),),name='input_1') input_2 = keras.Input(shape=(int(self.input_dim/2),),name='input_2') x_1 = layers.Dense(self.model_settings.layer_nodes[0], activation=None)(input_1)# x_2 = layers.Dense(self.model_settings.layer_nodes[0], activation=None)(input_2) for i in range(len(self.model_settings.layer_nodes)): x_1 = self.residual_block(x_1, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_2 = self.residual_block(x_2, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = layers.Subtract()([x_1, x_2]) for i in range(len(self.model_settings.layer_nodes)): x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) #O_A = keras.actiavation.tanh() outputs = layers.Dense(self.output_dim, activation='tanh')(x) self.model = keras.Model(inputs=(input_1,input_2), outputs=outputs) lr_schedule = keras.optimizers.schedules.ExponentialDecay( initial_learning_rate=self.model_settings.inital_learning_rate, decay_steps=1e6, decay_rate=self.model_settings.decay_rate, staircase=True ) model_optimiser = tf.keras.optimizers.Adam(learning_rate=lr_schedule) self.model.compile(optimizer=model_optimiser, loss=self.model_settings.loss_function, metrics=self.model_settings.metrics) self.model.load_weights(os.path.join(self.dir, 'model_weights.weights.h5'))
Set up a residual model with two inputs.
This method creates a model with two input branches, each with residual blocks, and combines them using subtraction.
def residual_model_4_input(self) ‑> None
-
Expand source code
def residual_model_4_input(self) -> None: """ Set up a residual model with four inputs. This method creates a model with four input branches, each with residual blocks, and combines them using subtraction. """ input_1 = keras.Input(shape=(int(self.input_dim/4),)) input_2 = keras.Input(shape=(int(self.input_dim/4),)) input_3 = keras.Input(shape=(int(self.input_dim/4),)) input_4 = keras.Input(shape=(int(self.input_dim/4),)) x_1 = layers.Dense(self.model_settings.layer_nodes[0], activation=None)(input_1) x_2 = layers.Dense(self.model_settings.layer_nodes[0], activation=None)(input_2) x_3 = layers.Dense(self.model_settings.layer_nodes[0], activation=None)(input_3) x_4 = layers.Dense(self.model_settings.layer_nodes[0], activation=None)(input_4) for i in range(len(self.model_settings.layer_nodes)): x_1 = self.residual_block(x_1, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_2 = self.residual_block(x_2, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_3 = self.residual_block(x_3, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_4 = self.residual_block(x_4, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_1 = layers.Subtract()([x_1, x_2]) x_3 = layers.Subtract()([x_3, x_4]) for i in range(len(self.model_settings.layer_nodes)): x_1 = self.residual_block(x_1, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_3 = self.residual_block(x_3, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = layers.Subtract()([x_1, x_3]) for i in range(len(self.model_settings.layer_nodes)): x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) outputs = layers.Dense(self.output_dim, activation=None)(x) self.model = keras.Model(inputs=[input_1,input_2,input_3,input_4], outputs=outputs) model_optimiser = tf.keras.optimizers.Adam(learning_rate=lr(self.model_settings.inital_learning_rate, self.model_settings.gamma_learning_rate, self.model_settings.power_learning_rate)) self.model.compile(optimizer=model_optimiser, loss=self.model_settings.loss_function, metrics=self.model_settings.metrics)
Set up a residual model with four inputs.
This method creates a model with four input branches, each with residual blocks, and combines them using subtraction.
def residual_model_8_input(self) ‑> None
-
Expand source code
def residual_model_8_input(self) -> None: """ Set up a residual model with eight inputs. This method creates a model with eight input branches, each with residual blocks, and combines them using subtraction. """ input_1 = keras.Input(shape=(int(self.input_dim/8),)) input_2 = keras.Input(shape=(int(self.input_dim/8),)) input_3 = keras.Input(shape=(int(self.input_dim/8),)) input_4 = keras.Input(shape=(int(self.input_dim/8),)) input_5 = keras.Input(shape=(int(self.input_dim/8),)) input_6 = keras.Input(shape=(int(self.input_dim/8),)) input_7 = keras.Input(shape=(int(self.input_dim/8),)) input_8 = keras.Input(shape=(int(self.input_dim/8),)) x_1 = layers.Dense(self.model_settings.layer_nodes[0], activation=None)(input_1) x_2 = layers.Dense(self.model_settings.layer_nodes[0], activation=None)(input_2) x_3 = layers.Dense(self.model_settings.layer_nodes[0], activation=None)(input_3) x_4 = layers.Dense(self.model_settings.layer_nodes[0], activation=None)(input_4) x_5 = layers.Dense(self.model_settings.layer_nodes[0], activation=None)(input_5) x_6 = layers.Dense(self.model_settings.layer_nodes[0], activation=None)(input_6) x_7 = layers.Dense(self.model_settings.layer_nodes[0], activation=None)(input_7) x_8 = layers.Dense(self.model_settings.layer_nodes[0], activation=None)(input_8) for i in range(len(self.model_settings.layer_nodes)): x_1 = self.residual_block(x_1, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_2 = self.residual_block(x_2, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_3 = self.residual_block(x_3, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_4 = self.residual_block(x_4, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_5 = self.residual_block(x_5, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_6 = self.residual_block(x_6, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_7 = self.residual_block(x_7, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_8 = self.residual_block(x_8, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_1 = layers.Subtract()([x_1, x_2]) x_3 = layers.Subtract()([x_3, x_4]) x_5 = layers.Subtract()([x_5, x_6]) x_7 = layers.Subtract()([x_7, x_8]) for i in range(len(self.model_settings.layer_nodes)): x_1 = self.residual_block(x_1, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_3 = self.residual_block(x_3, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_5 = self.residual_block(x_5, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_7 = self.residual_block(x_7, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_1 = layers.Subtract()([x_1, x_3]) x_5 = layers.Subtract()([x_5, x_7]) for i in range(len(self.model_settings.layer_nodes)): x_1 = self.residual_block(x_1, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_5 = self.residual_block(x_5, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = layers.Subtract()([x_1, x_5]) for i in range(len(self.model_settings.layer_nodes)): x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) outputs = layers.Dense(self.output_dim, activation=None)(x) self.model = keras.Model(inputs=[input_1,input_2,input_3,input_4,input_5,input_6,input_7,input_8], outputs=outputs) model_optimiser = tf.keras.optimizers.Adam(learning_rate=lr(self.model_settings.inital_learning_rate, self.model_settings.gamma_learning_rate, self.model_settings.power_learning_rate)) self.model.compile(optimizer=model_optimiser, loss=self.model_settings.loss_function, metrics=self.model_settings.metrics)
Set up a residual model with eight inputs.
This method creates a model with eight input branches, each with residual blocks, and combines them using subtraction.
def residual_model_existing(self, inputs: int = 1) ‑> None
-
Expand source code
def residual_model_existing(self, inputs: int =1) -> None: print('inputs: ', inputs) match inputs*2: case 1: self.residual_model_1_input() case 2: self.residual_model_2_input_existing() case 4: self.residual_model_4_input() case 8: self.residual_model_8_input() case _: self.residual_model_1_input()
def saveing(self, dir: str) ‑> None
-
Expand source code
def saveing(self, dir: str) -> None: """ Save the trained model to the specified directory. Args: dir (str): Directory to save the model. """ dir = os.path.join(dir, 'model.keras') self.model.save(dir)
Save the trained model to the specified directory.
Args
dir
:str
- Directory to save the model.
def saveing_existing(self, dir: str) ‑> None
-
Expand source code
def saveing_existing(self, dir: str) -> None: """ Save the trained model to the specified directory. Args: dir (str): Directory to save the model. """ files = str(len(os.listdir(dir)) + 1) print('Saving model to:', dir) dir = os.path.join(dir, 'model'+files+'.keras') self.model.save(dir)
Save the trained model to the specified directory.
Args
dir
:str
- Directory to save the model.
def setup_2input_residual_model(self) ‑> None
-
Expand source code
def setup_2input_residual_model(self) -> None: """ Set up a residual model with two inputs. This method creates a model with two input branches, each with residual blocks, and combines them using subtraction. """ input_1 = keras.Input(shape=(int(self.input_dim/2),)) input_2 = keras.Input(shape=(int(self.input_dim/2),)) x_1 = layers.Dense(self.model_settings.layer_nodes[0], activation=None)(input_1) x_2 = layers.Dense(self.model_settings.layer_nodes[0], activation=None)(input_2) x_1 = self.residual_block(x_1, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_1 = self.residual_block(x_1, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_1 = self.residual_block(x_1, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_1 = self.residual_block(x_1, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_1 = self.residual_block(x_1, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_1 = self.residual_block(x_1, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_1 = self.residual_block(x_1, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_1 = self.residual_block(x_1, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_2 = self.residual_block(x_2, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_2 = self.residual_block(x_2, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_2 = self.residual_block(x_2, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_2 = self.residual_block(x_2, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_2 = self.residual_block(x_2, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_2 = self.residual_block(x_2, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_2 = self.residual_block(x_2, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x_2 = self.residual_block(x_2, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = layers.Subtract()([x_1, x_2]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) outputs = layers.Dense(self.output_dim, activation=None)(x) self.model = keras.Model(inputs=[input_1,input_2], outputs=outputs) model_optimiser = tf.keras.optimizers.Adam(learning_rate=lr(self.model_settings.inital_learning_rate, self.model_settings.gamma_learning_rate, self.model_settings.power_learning_rate)) self.model.compile(optimizer=model_optimiser, loss=self.model_settings.loss_function, metrics=self.model_settings.metrics)
Set up a residual model with two inputs.
This method creates a model with two input branches, each with residual blocks, and combines them using subtraction.
def setup_model(self) ‑> None
-
Expand source code
def setup_model(self) -> None: """ Set up a standard sequential model based on the provided model settings. This method adds layers to the model, including dense layers, activation functions, batch normalization, and dropout layers. """ for i, nodes in enumerate(self.model_settings.layer_nodes): if i == 0: self.model.add(tf.keras.layers.Dense(nodes, kernel_initializer=self.model_settings.initializer, kernel_regularizer=self.model_settings.regularization)) self.model.add(tf.keras.layers.Activation(self.model_settings.activation)) if self.model_settings.batch_norm: self.model.add(tf.keras.layers.BatchNormalization()) if len(self.model_settings.dropout) != 0: self.model.add(tf.keras.layers.Dropout(self.model_settings.dropout[i])) else: self.model.add(tf.keras.layers.Dense(nodes, kernel_initializer=self.model_settings.initializer, kernel_regularizer=self.model_settings.regularization)) self.model.add(tf.keras.layers.Activation(self.model_settings.activation)) if self.model_settings.batch_norm: self.model.add(tf.keras.layers.BatchNormalization()) if len(self.model_settings.dropout) != 0: self.model.add(tf.keras.layers.Dropout(self.model_settings.dropout[i])) self.model.add(tf.keras.layers.Dense(self.output_dim, activation='tanh')) lr_schedule = keras.optimizers.schedules.ExponentialDecay( initial_learning_rate=self.model_settings.inital_learning_rate, decay_steps=1e6, decay_rate=self.model_settings.decay_rate, staircase=True ) model_optimiser = tf.keras.optimizers.Adam(learning_rate=lr_schedule) self.model.compile(optimizer=model_optimiser, loss=self.model_settings.loss_function, metrics=self.model_settings.metrics)
Set up a standard sequential model based on the provided model settings.
This method adds layers to the model, including dense layers, activation functions, batch normalization, and dropout layers.
def setup_model_existing(self) ‑> None
-
Expand source code
def setup_model_existing(self) -> None: """ Set up a standard sequential model based on the provided model settings. This method adds layers to the model, including dense layers, activation functions, batch normalization, and dropout layers. """ for i, nodes in enumerate(self.model_settings.layer_nodes): if i == 0: self.model.add(tf.keras.layers.Dense(nodes, kernel_initializer=self.model_settings.initializer, kernel_regularizer=self.model_settings.regularization)) self.model.add(tf.keras.layers.Activation(self.model_settings.activation)) if self.model_settings.batch_norm: self.model.add(tf.keras.layers.BatchNormalization()) if len(self.model_settings.dropout) != 0: self.model.add(tf.keras.layers.Dropout(self.model_settings.dropout[i])) else: self.model.add(tf.keras.layers.Dense(nodes, kernel_initializer=self.model_settings.initializer, kernel_regularizer=self.model_settings.regularization)) self.model.add(tf.keras.layers.Activation(self.model_settings.activation)) if self.model_settings.batch_norm: self.model.add(tf.keras.layers.BatchNormalization()) if len(self.model_settings.dropout) != 0: self.model.add(tf.keras.layers.Dropout(self.model_settings.dropout[i])) self.model.add(tf.keras.layers.Dense(self.output_dim, activation='tanh')) lr_schedule = keras.optimizers.schedules.ExponentialDecay( initial_learning_rate=self.model_settings.inital_learning_rate, decay_steps=1e6, decay_rate=self.model_settings.decay_rate, staircase=True ) model_optimiser = tf.keras.optimizers.Adam(learning_rate=lr_schedule) self.model.compile(optimizer=model_optimiser, loss=self.model_settings.loss_function, metrics=self.model_settings.metrics) self.model.load_weights(os.path.join(self.dir, 'model_weights.weights.h5'))
Set up a standard sequential model based on the provided model settings.
This method adds layers to the model, including dense layers, activation functions, batch normalization, and dropout layers.
def setup_residual_model(self) ‑> None
-
Expand source code
def setup_residual_model(self) -> None: """ Set up a residual model with a single input. This method creates a model with multiple residual blocks and compiles it. """ inputs = keras.Input(shape=(self.input_dim,)) x = layers.Dense(self.model_settings.layer_nodes[0], activation=None)(inputs) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) x = self.residual_block(x, nodes=self.model_settings.layer_nodes[0], n_layers=len(self.model_settings.layer_nodes), activation=self.model_settings.activation, dropout=self.model_settings.dropout[0]) outputs = layers.Dense(self.output_dim, activation=None)(x) self.model = keras.Model(inputs=inputs, outputs=outputs) model_optimiser = tf.keras.optimizers.Adam(learning_rate=lr(self.model_settings.inital_learning_rate, self.model_settings.gamma_learning_rate, self.model_settings.power_learning_rate)) self.model.compile(optimizer=model_optimiser, loss=self.model_settings.loss_function, metrics=self.model_settings.metrics)
Set up a residual model with a single input.
This method creates a model with multiple residual blocks and compiles it.
class lr (initial_learning_rate: float, gamma: float, power: float)
-
Expand source code
@src.saving.register_keras_serializable() class lr(tf.keras.optimizers.schedules.LearningRateSchedule): """ A custom learning rate schedule for TensorFlow/Keras. This schedule adjusts the learning rate based on the training step using a power decay formula. """ def __init__(self, initial_learning_rate: float, gamma: float, power: float) -> None: """ Initialize the learning rate schedule. Args: initial_learning_rate (float): Initial learning rate. gamma (float): Decay rate. power (float): Power for the decay formula. """ self.initial_learning_rate = initial_learning_rate self.gamma = gamma self.power = power def __call__(self, step: int) -> float: """ Calculate the learning rate for a given training step. Args: step (int): The current training step. Returns: float: The calculated learning rate. """ return self.initial_learning_rate * tf.pow((step * self.gamma +1), -self.power) def get_config(self) -> dict: """ Get the configuration of the learning rate schedule. Returns: dict: A dictionary containing the configuration. """ config = { 'initial_learning_rate': self.initial_learning_rate, 'gamma': self.gamma, 'power': self.power } return config
A custom learning rate schedule for TensorFlow/Keras.
This schedule adjusts the learning rate based on the training step using a power decay formula.
Initialize the learning rate schedule.
Args
initial_learning_rate
:float
- Initial learning rate.
gamma
:float
- Decay rate.
power
:float
- Power for the decay formula.
Ancestors
- keras.src.optimizers.schedules.learning_rate_schedule.LearningRateSchedule
Methods
def get_config(self) ‑> dict
-
Expand source code
def get_config(self) -> dict: """ Get the configuration of the learning rate schedule. Returns: dict: A dictionary containing the configuration. """ config = { 'initial_learning_rate': self.initial_learning_rate, 'gamma': self.gamma, 'power': self.power } return config
Get the configuration of the learning rate schedule.
Returns
dict
- A dictionary containing the configuration.