ise.models.predictors

ise.models.predictors.deep_ensemble

class ise.models.predictors.deep_ensemble.DeepEnsemble(ensemble_members=None, input_size=83, output_size=1, num_ensemble_members=3, output_sequence_length=86, latent_dim=1)

Bases: Module

Deep Ensemble Model using multiple LSTMs for time series forecasting.

This class implements an ensemble of LSTM-based predictors. Each LSTM model is trained separately, and predictions from all ensemble members are aggregated to provide a mean prediction along with an epistemic uncertainty estimate.

input_size

Size of the input features.

Type:

int

output_size

Size of the output features.

Type:

int

output_sequence_length

Length of the predicted output sequence.

Type:

int

loss_choices

List of loss functions used for different ensemble members.

Type:

list

ensemble_members

List of LSTM models used as ensemble members.

Type:

list

trained

Indicates whether all ensemble members have been trained.

Type:

bool

Parameters:

• ensemble_members (list, optional) - Pretrained LSTM models. If None, a new ensemble is created.

• input_size (int) - Number of input features.

• output_size (int) - Number of output features.

• num_ensemble_members (int) - Number of ensemble members to create if ensemble_members is None.

• output_sequence_length (int) - Length of the output sequence to predict.

• latent_dim (int) - Additional latent dimension added to the input.

Raises:

ValueError - If ensemble_members is provided but does not contain only LSTM instances.

fit(X, y, X_val=None, y_val=None, save_checkpoints=True, checkpoint_path='checkpoint_ensemble', early_stopping=False, epochs=100, batch_size=128, sequence_length=5, patience=10, verbose=True)

Trains each ensemble member on the provided data.

The ensemble members are trained separately, allowing for independent learning dynamics. Checkpoints can be saved for each model, and early stopping is available to prevent overfitting.

Parameters:

• X (Tensor) - Training input data.

• y (Tensor) - Training target data.

• X_val (Tensor, optional) - Validation input data for early stopping.

• y_val (Tensor, optional) - Validation target data for early stopping.

• save_checkpoints (bool, optional) - Whether to save checkpoints during training. Defaults to True.

• checkpoint_path (str, optional) - Path prefix for saving model checkpoints.

• early_stopping (bool, optional) - Whether to use early stopping. Defaults to False.

• epochs (int, optional) - Number of training epochs. Defaults to 100.

• batch_size (int, optional) - Batch size for training. Defaults to 128.

• sequence_length (int, optional) - Length of input sequences. Defaults to 5.

• patience (int, optional) - Number of epochs to wait before early stopping. Defaults to 10.

• verbose (bool, optional) - Whether to print training progress. Defaults to True.

Raises:

Warning - If the model has already been trained, a warning is issued before proceeding.

forward(x)

Performs a forward pass through the ensemble, aggregating predictions.

Each ensemble member makes a prediction, and the mean and standard deviation of these predictions are computed to provide an estimate of epistemic uncertainty.

Parameters:

x (Tensor) - Input tensor of shape (batch_size, sequence_length, input_size).

Returns:

• Mean prediction across all ensemble members.

• Epistemic uncertainty (standard deviation of predictions).

Return type:

Tuple[Tensor, Tensor]

Warning

• If the model is not trained, a warning is issued indicating that predictions may be unreliable.

classmethod load(model_path)

Loads a trained ensemble model from a file.

This method restores the ensemble’s state, including the metadata and individual LSTM members. The ensemble members are reinitialized and their state dictionaries are loaded from disk.

Parameters:

model_path (str) - Path to the saved model file.

Returns:

An instance of the loaded ensemble model.

Return type:

DeepEnsemble

Raises:

• FileNotFoundError - If any ensemble member’s file is missing.

• ValueError - If the saved model type does not match DeepEnsemble.

Notes

• The method ensures compatibility between the saved metadata and the loaded model.

• Loss functions are restored using a predefined lookup.

• The model is set to evaluation mode after loading.

predict(x)

Makes predictions using the trained ensemble.

This method calls forward while ensuring the model is in evaluation mode.

Parameters:

x (Tensor) - Input tensor for prediction.

Returns:

• Mean predictions across ensemble members.

• Uncertainty estimates (standard deviation of predictions).

Return type:

Tuple[Tensor, Tensor]

save(model_path)

Saves the ensemble model and its metadata.

This method stores the model parameters, metadata, and each ensemble member’s state dictionary. The metadata includes information about the ensemble members, such as their architecture, loss function, and training status.

Parameters:

model_path (str) - File path to save the model.

Raises:

ValueError - If attempting to save the model before it has been trained.

Notes

• The model directory is automatically created if it does not exist.

• Each ensemble member is saved in a separate subdirectory.

• After saving, any temporary checkpoint files are removed.

ise.models.predictors.lstm

class ise.models.predictors.lstm.LSTM(lstm_num_layers, lstm_hidden_size, input_size=83, output_size=1, criterion=MSELoss(), output_sequence_length=86, optimizer=<class 'torch.optim.adam.Adam'>)

Bases: Module

Long Short-Term Memory (LSTM) model for time series forecasting.

This class implements an LSTM network with multiple layers, dropout, and fully connected layers to generate predictions for sequential data.

lstm_num_layers

Number of LSTM layers in the model.

Type:

int

lstm_num_hidden

Number of hidden units in each LSTM layer.

Type:

int

input_size

Number of input features.

Type:

int

output_size

Number of output features.

Type:

int

output_sequence_length

Number of time steps predicted by the model.

Type:

int

device

Device on which the model runs (‘cuda’ or ‘cpu’).

Type:

str

lstm

LSTM layer for sequence modeling.

Type:

nn.LSTM

relu

ReLU activation function.

Type:

nn.ReLU

linear1

Intermediate fully connected layer.

Type:

nn.Linear

linear_out

Output layer mapping to final predictions.

Type:

nn.Linear

optimizer

Optimization algorithm used for training.

Type:

torch.optim.Optimizer

dropout

Dropout layer to prevent overfitting.

Type:

nn.Dropout

criterion

Loss function used for training.

Type:

torch.nn.modules.loss._Loss

trained

Flag indicating whether the model has been trained.

Type:

bool

Parameters:

• lstm_num_layers (int) - Number of LSTM layers.

• lstm_hidden_size (int) - Number of hidden units in each LSTM layer.

• input_size (int, optional) - Number of input features. Defaults to 83.

• output_size (int, optional) - Number of output features. Defaults to 1.

• criterion (torch.nn.modules.loss._Loss, optional) - Loss function. Defaults to MSELoss.

• output_sequence_length (int, optional) - Number of output time steps. Defaults to 86.

• optimizer (torch.optim.Optimizer, optional) - Optimizer type. Defaults to Adam.

fit(X, y, epochs=100, sequence_length=5, batch_size=64, criterion=None, X_val=None, y_val=None, save_checkpoints=True, checkpoint_path='checkpoint.pt', early_stopping=False, patience=10, verbose=True, dataclass=<class 'ise.data.dataclasses.EmulatorDataset'>)

Trains the LSTM model on the provided data.

Supports optional checkpointing and early stopping. If a checkpoint exists, training resumes from the last saved state.

Parameters:

• X (Tensor or DataFrame) - Input training data.

• y (Tensor or DataFrame) - Target values corresponding to the input data.

• epochs (int, optional) - Number of epochs for training. Defaults to 100.

• sequence_length (int, optional) - Length of input sequences. Defaults to 5.

• batch_size (int, optional) - Batch size used in training. Defaults to 64.

• criterion (torch.nn.modules.loss._Loss, optional) - Loss function. Defaults to None.

• X_val (Tensor or DataFrame, optional) - Validation input data. Defaults to None.

• y_val (Tensor or DataFrame, optional) - Validation target data. Defaults to None.

• save_checkpoints (bool, optional) - Whether to save model checkpoints. Defaults to True.

• checkpoint_path (str, optional) - Path to save model checkpoints. Defaults to ‘checkpoint.pt’.

• early_stopping (bool, optional) - Whether to enable early stopping. Defaults to False.

• patience (int, optional) - Number of epochs to wait before stopping. Defaults to 10.

• verbose (bool, optional) - Whether to print training progress. Defaults to True.

• dataclass (type, optional) - Dataset class for handling data. Defaults to EmulatorDataset.

Raises:

ValueError - If no loss function is provided.

Notes

• If validation data is provided but early stopping is disabled, a warning is issued.

• If a checkpoint exists, training resumes from the saved epoch.

• If early stopping is enabled, the model stops training when validation loss stops improving.

forward(x)

Performs a forward pass through the LSTM network.

Given an input sequence, the LSTM processes the sequence to extract features, which are passed through a fully connected network to generate predictions.

Parameters:

x (Tensor) - Input tensor of shape (batch_size, sequence_length, input_size).

Returns:

Output tensor of shape (batch_size, output_size), representing the model’s predictions.

Return type:

Tensor

predict(X, sequence_length=5, batch_size=64, dataclass=<class 'ise.data.dataclasses.EmulatorDataset'>)

Generates predictions using the trained LSTM model.

The model processes input sequences and returns predictions. Predictions are computed in a batch-wise manner to optimize memory usage.

Parameters:

• X (Tensor or DataFrame) - Input data for prediction.

• sequence_length (int, optional) - Length of input sequences. Defaults to 5.

• batch_size (int, optional) - Batch size used for inference. Defaults to 64.

• dataclass (type, optional) - Dataset class for handling data. Defaults to EmulatorDataset.

Returns:

Predicted values for the input data.

Return type:

Tensor

Notes

• The model is set to evaluation mode before making predictions.

• Data is converted to tensors if initially provided as pandas DataFrames.