19  Hyperparameter Tuning: VBDP

In this tutorial, we will show how spotPython can be integrated into the PyTorch training workflow for a classifiaction task.

Note

Ensure that the correspondiing data is available as ./data/VBDP/train.csv.

This document refers to the following software versions:

pip list | grep  "spot[RiverPython]"
spotPython                 0.2.31
spotRiver                  0.0.93
Note: you may need to restart the kernel to use updated packages.

spotPython can be installed via pip. Alternatively, the source code can be downloaded from gitHub: https://github.com/sequential-parameter-optimization/spotPython.

!pip install spotPython
# import sys
# !{sys.executable} -m pip install --upgrade build
# !{sys.executable} -m pip install --upgrade --force-reinstall spotPython

19.1 Setup

Before we consider the detailed experimental setup, we select the parameters that affect run time, initial design size and the device that is used.

MAX_TIME = 60
INIT_SIZE = 10
DEVICE = None # "cpu" # "cuda:0"
from spotPython.utils.device import getDevice
DEVICE = getDevice(DEVICE)
print(DEVICE)
mps
import os
import copy
import socket
from datetime import datetime
from dateutil.tz import tzlocal
start_time = datetime.now(tzlocal())
HOSTNAME = socket.gethostname().split(".")[0]
experiment_name = '25-torch' + "_" + HOSTNAME + "_" + str(MAX_TIME) + "min_" + str(INIT_SIZE) + "init_" + str(start_time).split(".", 1)[0].replace(' ', '_')
experiment_name = experiment_name.replace(':', '-')
print(experiment_name)
if not os.path.exists('./figures'):
    os.makedirs('./figures')
25-torch_p040025_60min_10init_2023-06-16_15-40-33

19.2 Initialization of the fun_control Dictionary

spotPython uses a Python dictionary for storing the information required for the hyperparameter tuning process, which was described in Section 13.2.

from spotPython.utils.init import fun_control_init
fun_control = fun_control_init(task="classification",
    tensorboard_path="runs/25_spot_torch_vbdp",
    device=DEVICE)

20 PyTorch Data Loading

20.1 1. Load VBDP Data

import pandas as pd
from sklearn.preprocessing import OrdinalEncoder
train_df = pd.read_csv('./data/VBDP/train.csv')
# remove the id column
train_df = train_df.drop(columns=['id'])
n_samples = train_df.shape[0]
n_features = train_df.shape[1] - 1
target_column = "prognosis"
# # Encoder our prognosis labels as integers for easier decoding later
enc = OrdinalEncoder()
train_df[target_column] = enc.fit_transform(train_df[[target_column]])
train_df.head()

# convert all entries to int for faster processing
train_df = train_df.astype(int)
from spotPython.data.vbdp import combine_features
df_new = train_df.copy()
# save the target column using "target_column" as the column name
target = train_df[target_column]
# remove the target column
df_new = df_new.drop(columns=[target_column])
train_df = combine_features(df_new)
# add the target column back
train_df[target_column] = target
train_df.head()
sudden_fever headache mouth_bleed nose_bleed muscle_pain joint_pain vomiting rash diarrhea hypotension ... 6039 6040 6041 6042 6043 6044 6045 6046 6047 prognosis
0 1 1 0 1 1 1 1 0 1 1 ... 0 0 0 0 0 0 0 0 0 3
1 0 0 0 0 0 0 1 0 1 0 ... 0 0 0 0 0 0 0 0 0 7
2 0 1 1 1 0 1 1 1 1 1 ... 1 1 0 1 1 0 1 1 0 3
3 0 0 1 1 1 1 0 1 0 1 ... 0 0 0 0 0 0 0 0 0 10
4 0 0 0 0 0 0 0 0 1 0 ... 0 1 1 0 1 1 0 0 0 6

5 rows × 6113 columns

  • feature engineering: 6112 features
from sklearn.model_selection import train_test_split
import numpy as np

n_samples = train_df.shape[0]
n_features = train_df.shape[1] - 1
train_df.columns = [f"x{i}" for i in range(1, n_features+1)] + [target_column]
X_train, X_test, y_train, y_test = train_test_split(train_df.drop(target_column, axis=1), train_df[target_column],
                                                    random_state=42,
                                                    test_size=0.25,
                                                    stratify=train_df[target_column])
trainset = pd.DataFrame(np.hstack((X_train, np.array(y_train).reshape(-1, 1))))
testset = pd.DataFrame(np.hstack((X_test, np.array(y_test).reshape(-1, 1))))
trainset.columns = [f"x{i}" for i in range(1, n_features+1)] + [target_column]
testset.columns = [f"x{i}" for i in range(1, n_features+1)] + [target_column]
print(train_df.shape)
print(trainset.shape)
print(testset.shape)
(707, 6113)
(530, 6113)
(177, 6113)
import torch
from sklearn.model_selection import train_test_split
from spotPython.torch.dataframedataset import DataFrameDataset
dtype_x = torch.float32
dtype_y = torch.long
train_df = DataFrameDataset(train_df, target_column=target_column, dtype_x=dtype_x, dtype_y=dtype_y)
train = DataFrameDataset(trainset, target_column=target_column, dtype_x=dtype_x, dtype_y=dtype_y)
test = DataFrameDataset(testset, target_column=target_column, dtype_x=dtype_x, dtype_y=dtype_y)
n_samples = len(train)
# add the dataset to the fun_control
fun_control.update({"data": train_df, # full dataset,
               "train": train,
               "test": test,
               "n_samples": n_samples,
               "target_column": target_column})

20.2 The Model (Algorithm) to be Tuned

20.3 Specification of the Preprocessing Model

After the training and test data are specified and added to the fun_control dictionary, spotPython allows the specification of a data preprocessing pipeline, e.g., for the scaling of the data or for the one-hot encoding of categorical variables, see Section 13.4.1. This feature is not used here, so we do not change the default value (which is None).

20.4 Select algorithm and core_model_hyper_dict

20.4.1 Implementing a Configurable Neural Network With spotPython

spotPython includes the Net_vbdp class which is implemented in the file netvbdp.py. The class is imported here.

This class inherits from the class Net_Core which is implemented in the file netcore.py, see ?sec-the-net-core-class-24.

21 add the nn model to the fun_control dictionary

from spotPython.torch.netvbdp import Net_vbdp
from spotPython.data.torch_hyper_dict import TorchHyperDict
from spotPython.hyperparameters.values import add_core_model_to_fun_control
fun_control = add_core_model_to_fun_control(core_model=Net_vbdp,
                              fun_control=fun_control,
                              hyper_dict=TorchHyperDict)

21.1 Modifying the Hyperparameters

spotPython provides functions for modifying the hyperparameters, their bounds and factors as well as for activating and de-activating hyperparameters without re-compilation of the Python source code. These functions were described in Section 13.5.3.

Small number of epochs for demonstration purposes
  • epochs is set to 2 and 3 for demonstration purposes. These values are too small for a real application.
from spotPython.hyperparameters.values import modify_hyper_parameter_bounds

fun_control = modify_hyper_parameter_bounds(fun_control, "_L0", bounds=[n_features, n_features])
fun_control = modify_hyper_parameter_bounds(fun_control, "l1", bounds=[6, 13])
fun_control = modify_hyper_parameter_bounds(fun_control, "epochs", bounds=[2, 2])
fun_control = modify_hyper_parameter_bounds(fun_control, "patience", bounds=[2, 6])
fun_control = modify_hyper_parameter_bounds(fun_control, "lr_mult", bounds=[1e-3, 1e-3])
fun_control = modify_hyper_parameter_bounds(fun_control, "sgd_momentum", bounds=[0.9, 0.9])
from spotPython.hyperparameters.values import modify_hyper_parameter_levels
fun_control = modify_hyper_parameter_levels(fun_control, "optimizer",["Adam", "AdamW", "Adamax", "NAdam"])
# fun_control = modify_hyper_parameter_levels(fun_control, "optimizer", ["Adam"])
# fun_control = modify_hyper_parameter_levels(fun_control, "leaf_model", ["LinearRegression"])
# fun_control["core_model_hyper_dict"]
fun_control = modify_hyper_parameter_bounds(fun_control,
    "lr_mult", bounds=[1e-3, 1e-3])
fun_control = modify_hyper_parameter_bounds(fun_control,
    "sgd_momentum", bounds=[0.9, 0.9])

21.1.1 Optimizers

Optimizers are described in Section 13.6.

21.2 Evaluation

The evaluation procedure requires the specification of two elements:

  1. the way how the data is split into a train and a test set (see Section 13.7)
  2. the loss function (and a metric).

21.2.1 Loss Functions and Metrics

The loss function is specified by the key "loss_function". We will use CrossEntropy loss for the multiclass-classification task.

from torch.nn import CrossEntropyLoss
loss_function = CrossEntropyLoss()
fun_control.update({"loss_function": loss_function})

21.2.2 Metric

  • We will use the MAP@k metric for the evaluation of the model. Here is an example how this metric is calculated.
from spotPython.torch.mapk import MAPK
import torch
mapk = MAPK(k=2)
target = torch.tensor([0, 1, 2, 2])
preds = torch.tensor(
    [
        [0.5, 0.2, 0.2],  # 0 is in top 2
        [0.3, 0.4, 0.2],  # 1 is in top 2
        [0.2, 0.4, 0.3],  # 2 is in top 2
        [0.7, 0.2, 0.1],  # 2 isn't in top 2
    ]
)  
mapk.update(preds, target)
print(mapk.compute()) # tensor(0.6250)
tensor(0.6250)
from spotPython.torch.mapk import MAPK
import torchmetrics
metric_torch = MAPK(k=3)
fun_control.update({"metric_torch": metric_torch})

21.3 Preparing the SPOT Call

The following code passes the information about the parameter ranges and bounds to spot.

# extract the variable types, names, and bounds
from spotPython.hyperparameters.values import (get_bound_values,
    get_var_name,
    get_var_type,)
var_type = get_var_type(fun_control)
var_name = get_var_name(fun_control)
fun_control.update({"var_type": var_type,
                    "var_name": var_name})
lower = get_bound_values(fun_control, "lower")
upper = get_bound_values(fun_control, "upper")

Now, the dictionary fun_control contains all information needed for the hyperparameter tuning. Before the hyperparameter tuning is started, it is recommended to take a look at the experimental design. The method gen_design_table generates a design table as follows:

from spotPython.utils.eda import gen_design_table
print(gen_design_table(fun_control))
| name         | type   | default   |    lower |    upper | transform             |
|--------------|--------|-----------|----------|----------|-----------------------|
| _L0          | int    | 64        | 6112     | 6112     | None                  |
| l1           | int    | 8         |    6     |   13     | transform_power_2_int |
| dropout_prob | float  | 0.01      |    0     |    0.9   | None                  |
| lr_mult      | float  | 1.0       |    0.001 |    0.001 | None                  |
| batch_size   | int    | 4         |    1     |    4     | transform_power_2_int |
| epochs       | int    | 4         |    2     |    2     | transform_power_2_int |
| k_folds      | int    | 1         |    1     |    1     | None                  |
| patience     | int    | 2         |    2     |    6     | transform_power_2_int |
| optimizer    | factor | SGD       |    0     |    3     | None                  |
| sgd_momentum | float  | 0.0       |    0.9   |    0.9   | None                  |

This allows to check if all information is available and if the information is correct.

21.4 The Objective Function fun_torch

The objective function fun_torch is selected next. It implements an interface from PyTorch’s training, validation, and testing methods to spotPython.

from spotPython.fun.hypertorch import HyperTorch
fun = HyperTorch().fun_torch
from spotPython.hyperparameters.values import get_default_hyperparameters_as_array
hyper_dict=TorchHyperDict().load()
X_start = get_default_hyperparameters_as_array(fun_control, hyper_dict)

21.5 Starting the Hyperparameter Tuning

The spotPython hyperparameter tuning is started by calling the Spot function as described in Section 13.12.

import numpy as np
from spotPython.spot import spot
from math import inf
spot_tuner = spot.Spot(fun=fun,
                   lower = lower,
                   upper = upper,
                   fun_evals = inf,
                   fun_repeats = 1,
                   max_time = MAX_TIME,
                   noise = False,
                   tolerance_x = np.sqrt(np.spacing(1)),
                   var_type = var_type,
                   var_name = var_name,
                   infill_criterion = "y",
                   n_points = 1,
                   seed=123,
                   log_level = 50,
                   show_models= False,
                   show_progress= True,
                   fun_control = fun_control,
                   design_control={"init_size": INIT_SIZE,
                                   "repeats": 1},
                   surrogate_control={"noise": True,
                                      "cod_type": "norm",
                                      "min_theta": -4,
                                      "max_theta": 3,
                                      "n_theta": len(var_name),
                                      "model_fun_evals": 10_000,
                                      "log_level": 50
                                      })
spot_tuner.run(X_start=X_start)

config: {'_L0': 6112, 'l1': 1024, 'dropout_prob': 0.26515610830779995, 'lr_mult': 0.001, 'batch_size': 8, 'epochs': 4, 'k_folds': 1, 'patience': 32, 'optimizer': 'Adamax', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.397894585574115
Accuracy on hold-out set: 0.07547169811320754
MAPK value on hold-out data: 0.15663579106330872
Epoch: 2
Loss on hold-out set: 2.397868695082488
Accuracy on hold-out set: 0.07075471698113207
MAPK value on hold-out data: 0.15740741789340973
Epoch: 3
Loss on hold-out set: 2.397852385485614
Accuracy on hold-out set: 0.06132075471698113
MAPK value on hold-out data: 0.14120371639728546
Epoch: 4
Loss on hold-out set: 2.3978051167947276
Accuracy on hold-out set: 0.07075471698113207
MAPK value on hold-out data: 0.16280867159366608
Returned to Spot: Validation loss: 2.3978051167947276
----------------------------------------------

config: {'_L0': 6112, 'l1': 256, 'dropout_prob': 0.6368989501775649, 'lr_mult': 0.001, 'batch_size': 4, 'epochs': 4, 'k_folds': 1, 'patience': 32, 'optimizer': 'Adamax', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3993411289071136
Accuracy on hold-out set: 0.08490566037735849
MAPK value on hold-out data: 0.13522012531757355
Epoch: 2
Loss on hold-out set: 2.3994905633746453
Accuracy on hold-out set: 0.09905660377358491
MAPK value on hold-out data: 0.13757862150669098
Epoch: 3
Loss on hold-out set: 2.3992917177812108
Accuracy on hold-out set: 0.10849056603773585
MAPK value on hold-out data: 0.14386792480945587
Epoch: 4
Loss on hold-out set: 2.399499330880507
Accuracy on hold-out set: 0.08962264150943396
MAPK value on hold-out data: 0.1328616440296173
Returned to Spot: Validation loss: 2.399499330880507
----------------------------------------------

config: {'_L0': 6112, 'l1': 512, 'dropout_prob': 0.06795318590380832, 'lr_mult': 0.001, 'batch_size': 4, 'epochs': 4, 'k_folds': 1, 'patience': 8, 'optimizer': 'Adamax', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3974383777042605
Accuracy on hold-out set: 0.08962264150943396
MAPK value on hold-out data: 0.18160375952720642
Epoch: 2
Loss on hold-out set: 2.397403069262235
Accuracy on hold-out set: 0.08962264150943396
MAPK value on hold-out data: 0.18396224081516266
Epoch: 3
Loss on hold-out set: 2.397404229865884
Accuracy on hold-out set: 0.08962264150943396
MAPK value on hold-out data: 0.18317607045173645
Epoch: 4
Loss on hold-out set: 2.3973749763560743
Accuracy on hold-out set: 0.08962264150943396
MAPK value on hold-out data: 0.18474839627742767
Returned to Spot: Validation loss: 2.3973749763560743
----------------------------------------------

config: {'_L0': 6112, 'l1': 128, 'dropout_prob': 0.456533346732735, 'lr_mult': 0.001, 'batch_size': 4, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3984526058412947
Accuracy on hold-out set: 0.08490566037735849
MAPK value on hold-out data: 0.15408805012702942
Epoch: 2
Loss on hold-out set: 2.3985123544369102
Accuracy on hold-out set: 0.08490566037735849
MAPK value on hold-out data: 0.1525157392024994
Epoch: 3
Loss on hold-out set: 2.398514302271717
Accuracy on hold-out set: 0.08490566037735849
MAPK value on hold-out data: 0.15408805012702942
Epoch: 4
Loss on hold-out set: 2.3985012792191416
Accuracy on hold-out set: 0.08490566037735849
MAPK value on hold-out data: 0.14937108755111694
Returned to Spot: Validation loss: 2.3985012792191416
----------------------------------------------

config: {'_L0': 6112, 'l1': 2048, 'dropout_prob': 0.435068640608333, 'lr_mult': 0.001, 'batch_size': 8, 'epochs': 4, 'k_folds': 1, 'patience': 32, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3976643262086093
Accuracy on hold-out set: 0.08018867924528301
MAPK value on hold-out data: 0.17283952236175537
Epoch: 2
Loss on hold-out set: 2.3976193798912897
Accuracy on hold-out set: 0.08490566037735849
MAPK value on hold-out data: 0.1766975373029709
Epoch: 3
Loss on hold-out set: 2.397466368145413
Accuracy on hold-out set: 0.08490566037735849
MAPK value on hold-out data: 0.17901237308979034
Epoch: 4
Loss on hold-out set: 2.397409545050727
Accuracy on hold-out set: 0.08962264150943396
MAPK value on hold-out data: 0.1766975373029709
Returned to Spot: Validation loss: 2.397409545050727
----------------------------------------------

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.3175980093998585, 'lr_mult': 0.001, 'batch_size': 4, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'NAdam', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.397276797384586
Accuracy on hold-out set: 0.1179245283018868
MAPK value on hold-out data: 0.2279873937368393
Epoch: 2
Loss on hold-out set: 2.3967610440164244
Accuracy on hold-out set: 0.1179245283018868
MAPK value on hold-out data: 0.22484277188777924
Epoch: 3
Loss on hold-out set: 2.3960829096020393
Accuracy on hold-out set: 0.12735849056603774
MAPK value on hold-out data: 0.23349057137966156
Epoch: 4
Loss on hold-out set: 2.3949763325025453
Accuracy on hold-out set: 0.1320754716981132
MAPK value on hold-out data: 0.2421383559703827
Returned to Spot: Validation loss: 2.3949763325025453
----------------------------------------------

config: {'_L0': 6112, 'l1': 128, 'dropout_prob': 0.10217787740156026, 'lr_mult': 0.001, 'batch_size': 8, 'epochs': 4, 'k_folds': 1, 'patience': 4, 'optimizer': 'Adam', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3971679475572376
Accuracy on hold-out set: 0.12735849056603774
MAPK value on hold-out data: 0.19367282092571259
Epoch: 2
Loss on hold-out set: 2.3971016848528826
Accuracy on hold-out set: 0.12735849056603774
MAPK value on hold-out data: 0.19367282092571259
Epoch: 3
Loss on hold-out set: 2.3971388693209046
Accuracy on hold-out set: 0.12735849056603774
MAPK value on hold-out data: 0.19367282092571259
Epoch: 4
Loss on hold-out set: 2.3970733130419695
Accuracy on hold-out set: 0.12735849056603774
MAPK value on hold-out data: 0.19367282092571259
Returned to Spot: Validation loss: 2.3970733130419695
----------------------------------------------

config: {'_L0': 6112, 'l1': 4096, 'dropout_prob': 0.5582660802134882, 'lr_mult': 0.001, 'batch_size': 8, 'epochs': 4, 'k_folds': 1, 'patience': 8, 'optimizer': 'NAdam', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3974198323708995
Accuracy on hold-out set: 0.1320754716981132
MAPK value on hold-out data: 0.21682098507881165
Epoch: 2
Loss on hold-out set: 2.3973477063355624
Accuracy on hold-out set: 0.14150943396226415
MAPK value on hold-out data: 0.21990740299224854
Epoch: 3
Loss on hold-out set: 2.397355883209794
Accuracy on hold-out set: 0.1320754716981132
MAPK value on hold-out data: 0.19675925374031067
Epoch: 4
Loss on hold-out set: 2.3972159579948142
Accuracy on hold-out set: 0.1509433962264151
MAPK value on hold-out data: 0.22067900002002716
Returned to Spot: Validation loss: 2.3972159579948142
----------------------------------------------

config: {'_L0': 6112, 'l1': 4096, 'dropout_prob': 0.8475085430864048, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 4, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.398350380501657
Accuracy on hold-out set: 0.08490566037735849
MAPK value on hold-out data: 0.14229559898376465
Epoch: 2
Loss on hold-out set: 2.3980596290444427
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.1666666716337204
Epoch: 3
Loss on hold-out set: 2.3984738655810087
Accuracy on hold-out set: 0.09905660377358491
MAPK value on hold-out data: 0.15566039085388184
Epoch: 4
Loss on hold-out set: 2.3985142752809345
Accuracy on hold-out set: 0.06132075471698113
MAPK value on hold-out data: 0.13679248094558716
Returned to Spot: Validation loss: 2.3985142752809345
----------------------------------------------

config: {'_L0': 6112, 'l1': 64, 'dropout_prob': 0.8086972970888939, 'lr_mult': 0.001, 'batch_size': 16, 'epochs': 4, 'k_folds': 1, 'patience': 64, 'optimizer': 'Adam', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3985851321901595
Accuracy on hold-out set: 0.07547169811320754
MAPK value on hold-out data: 0.1525297611951828
Epoch: 2
Loss on hold-out set: 2.3987087351935252
Accuracy on hold-out set: 0.08490566037735849
MAPK value on hold-out data: 0.1659226268529892
Epoch: 3
Loss on hold-out set: 2.398700935500009
Accuracy on hold-out set: 0.09433962264150944
MAPK value on hold-out data: 0.1569940596818924
Epoch: 4
Loss on hold-out set: 2.398722461291722
Accuracy on hold-out set: 0.08490566037735849
MAPK value on hold-out data: 0.158482164144516
Returned to Spot: Validation loss: 2.398722461291722
----------------------------------------------

config: {'_L0': 6112, 'l1': 4096, 'dropout_prob': 0.2354775707662662, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'NAdam', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.398008989837934
Accuracy on hold-out set: 0.03773584905660377
MAPK value on hold-out data: 0.12028304487466812
Epoch: 2
Loss on hold-out set: 2.3976206082218097
Accuracy on hold-out set: 0.08490566037735849
MAPK value on hold-out data: 0.15015725791454315
Epoch: 3
Loss on hold-out set: 2.397112153611093
Accuracy on hold-out set: 0.0660377358490566
MAPK value on hold-out data: 0.14465412497520447
Epoch: 4
Loss on hold-out set: 2.3964138278421365
Accuracy on hold-out set: 0.08490566037735849
MAPK value on hold-out data: 0.16352201998233795
Returned to Spot: Validation loss: 2.3964138278421365
----------------------------------------------
spotPython tuning: 2.3949763325025453 [----------] 0.31% 

config: {'_L0': 6112, 'l1': 4096, 'dropout_prob': 0.280312419027338, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'NAdam', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.397442145167657
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.16823899745941162
Epoch: 2
Loss on hold-out set: 2.3971147064892753
Accuracy on hold-out set: 0.09433962264150944
MAPK value on hold-out data: 0.17845913767814636
Epoch: 3
Loss on hold-out set: 2.3966588006829315
Accuracy on hold-out set: 0.08962264150943396
MAPK value on hold-out data: 0.1949685513973236
Epoch: 4
Loss on hold-out set: 2.396013370100057
Accuracy on hold-out set: 0.08962264150943396
MAPK value on hold-out data: 0.20440247654914856
Returned to Spot: Validation loss: 2.396013370100057
----------------------------------------------
spotPython tuning: 2.3949763325025453 [----------] 0.62% 

config: {'_L0': 6112, 'l1': 4096, 'dropout_prob': 0.35612419119329064, 'lr_mult': 0.001, 'batch_size': 16, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'NAdam', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3980847086225237
Accuracy on hold-out set: 0.0660377358490566
MAPK value on hold-out data: 0.1302083283662796
Epoch: 2
Loss on hold-out set: 2.397933874811445
Accuracy on hold-out set: 0.08018867924528301
MAPK value on hold-out data: 0.159226194024086
Epoch: 3
Loss on hold-out set: 2.397876875741141
Accuracy on hold-out set: 0.06132075471698113
MAPK value on hold-out data: 0.15550597012043
Epoch: 4
Loss on hold-out set: 2.3978610038757324
Accuracy on hold-out set: 0.05188679245283019
MAPK value on hold-out data: 0.1264880895614624
Returned to Spot: Validation loss: 2.3978610038757324
----------------------------------------------
spotPython tuning: 2.3949763325025453 [----------] 0.66% 

config: {'_L0': 6112, 'l1': 4096, 'dropout_prob': 0.34862581565527756, 'lr_mult': 0.001, 'batch_size': 4, 'epochs': 4, 'k_folds': 1, 'patience': 64, 'optimizer': 'NAdam', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3980167002048134
Accuracy on hold-out set: 0.08490566037735849
MAPK value on hold-out data: 0.1800314337015152
Epoch: 2
Loss on hold-out set: 2.3977928116636456
Accuracy on hold-out set: 0.08490566037735849
MAPK value on hold-out data: 0.18867924809455872
Epoch: 3
Loss on hold-out set: 2.3975073076644033
Accuracy on hold-out set: 0.08018867924528301
MAPK value on hold-out data: 0.19261007010936737
Epoch: 4
Loss on hold-out set: 2.3971244569094674
Accuracy on hold-out set: 0.08490566037735849
MAPK value on hold-out data: 0.22012577950954437
Returned to Spot: Validation loss: 2.3971244569094674
----------------------------------------------
spotPython tuning: 2.3949763325025453 [----------] 0.81% 

config: {'_L0': 6112, 'l1': 1024, 'dropout_prob': 0.3381737496819435, 'lr_mult': 0.001, 'batch_size': 4, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'NAdam', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3977291404076344
Accuracy on hold-out set: 0.09905660377358491
MAPK value on hold-out data: 0.18238992989063263
Epoch: 2
Loss on hold-out set: 2.397665761551767
Accuracy on hold-out set: 0.09433962264150944
MAPK value on hold-out data: 0.16823898255825043
Epoch: 3
Loss on hold-out set: 2.3975564623778722
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.1871069371700287
Epoch: 4
Loss on hold-out set: 2.397476173796744
Accuracy on hold-out set: 0.12264150943396226
MAPK value on hold-out data: 0.19811320304870605
Returned to Spot: Validation loss: 2.397476173796744
----------------------------------------------
spotPython tuning: 2.3949763325025453 [----------] 0.89% 

config: {'_L0': 6112, 'l1': 4096, 'dropout_prob': 0.32747667646720513, 'lr_mult': 0.001, 'batch_size': 4, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'NAdam', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3975240689403607
Accuracy on hold-out set: 0.09905660377358491
MAPK value on hold-out data: 0.17610062658786774
Epoch: 2
Loss on hold-out set: 2.3973306349988253
Accuracy on hold-out set: 0.08018867924528301
MAPK value on hold-out data: 0.16981133818626404
Epoch: 3
Loss on hold-out set: 2.397141641041018
Accuracy on hold-out set: 0.07547169811320754
MAPK value on hold-out data: 0.17610062658786774
Epoch: 4
Loss on hold-out set: 2.397028437200582
Accuracy on hold-out set: 0.07547169811320754
MAPK value on hold-out data: 0.16823898255825043
Returned to Spot: Validation loss: 2.397028437200582
----------------------------------------------
spotPython tuning: 2.3949763325025453 [----------] 1.03% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.26879366207150224, 'lr_mult': 0.001, 'batch_size': 4, 'epochs': 4, 'k_folds': 1, 'patience': 32, 'optimizer': 'NAdam', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3976370973407097
Accuracy on hold-out set: 0.16037735849056603
MAPK value on hold-out data: 0.22720122337341309
Epoch: 2
Loss on hold-out set: 2.397098478281273
Accuracy on hold-out set: 0.18867924528301888
MAPK value on hold-out data: 0.2759433686733246
Epoch: 3
Loss on hold-out set: 2.3965415009912454
Accuracy on hold-out set: 0.16037735849056603
MAPK value on hold-out data: 0.2704402506351471
Epoch: 4
Loss on hold-out set: 2.395292893895563
Accuracy on hold-out set: 0.2169811320754717
MAPK value on hold-out data: 0.3097483813762665
Returned to Spot: Validation loss: 2.395292893895563
----------------------------------------------
spotPython tuning: 2.3949763325025453 [----------] 1.36% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.8523853629491498, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'NAdam', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3977531869456454
Accuracy on hold-out set: 0.12264150943396226
MAPK value on hold-out data: 0.1863207370042801
Epoch: 2
Loss on hold-out set: 2.3974104107550853
Accuracy on hold-out set: 0.11320754716981132
MAPK value on hold-out data: 0.17610062658786774
Epoch: 3
Loss on hold-out set: 2.3972729242072917
Accuracy on hold-out set: 0.09433962264150944
MAPK value on hold-out data: 0.1745283007621765
Epoch: 4
Loss on hold-out set: 2.3966573859160802
Accuracy on hold-out set: 0.09433962264150944
MAPK value on hold-out data: 0.16981130838394165
Returned to Spot: Validation loss: 2.3966573859160802
----------------------------------------------
spotPython tuning: 2.3949763325025453 [----------] 1.99% 

config: {'_L0': 6112, 'l1': 4096, 'dropout_prob': 0.3129928408802308, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'NAdam', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3978051644451215
Accuracy on hold-out set: 0.1179245283018868
MAPK value on hold-out data: 0.1941823959350586
Epoch: 2
Loss on hold-out set: 2.3973899674865433
Accuracy on hold-out set: 0.12735849056603774
MAPK value on hold-out data: 0.20440249145030975
Epoch: 3
Loss on hold-out set: 2.3967591186739363
Accuracy on hold-out set: 0.1509433962264151
MAPK value on hold-out data: 0.23034584522247314
Epoch: 4
Loss on hold-out set: 2.396109059171857
Accuracy on hold-out set: 0.15566037735849056
MAPK value on hold-out data: 0.2350628525018692
Returned to Spot: Validation loss: 2.396109059171857
----------------------------------------------
spotPython tuning: 2.3949763325025453 [----------] 2.30% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.3202873049403221, 'lr_mult': 0.001, 'batch_size': 4, 'epochs': 4, 'k_folds': 1, 'patience': 8, 'optimizer': 'NAdam', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3974952247907533
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.19496850669384003
Epoch: 2
Loss on hold-out set: 2.3967573057930425
Accuracy on hold-out set: 0.1179245283018868
MAPK value on hold-out data: 0.20440247654914856
Epoch: 3
Loss on hold-out set: 2.39603435318425
Accuracy on hold-out set: 0.13679245283018868
MAPK value on hold-out data: 0.20990565419197083
Epoch: 4
Loss on hold-out set: 2.3945745432151937
Accuracy on hold-out set: 0.14622641509433962
MAPK value on hold-out data: 0.2154087871313095
Returned to Spot: Validation loss: 2.3945745432151937
----------------------------------------------
spotPython tuning: 2.3945745432151937 [----------] 2.63% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.36303595437578823, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'NAdam', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3972799328138246
Accuracy on hold-out set: 0.12264150943396226
MAPK value on hold-out data: 0.18553459644317627
Epoch: 2
Loss on hold-out set: 2.396546186141248
Accuracy on hold-out set: 0.11320754716981132
MAPK value on hold-out data: 0.22012574970722198
Epoch: 3
Loss on hold-out set: 2.3951884350686705
Accuracy on hold-out set: 0.12735849056603774
MAPK value on hold-out data: 0.22562891244888306
Epoch: 4
Loss on hold-out set: 2.393136930915545
Accuracy on hold-out set: 0.13679245283018868
MAPK value on hold-out data: 0.23034584522247314
Returned to Spot: Validation loss: 2.393136930915545
----------------------------------------------
spotPython tuning: 2.393136930915545 [----------] 3.31% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.40178776340101174, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'NAdam', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.397080848801811
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.2562892436981201
Epoch: 2
Loss on hold-out set: 2.3962906284152337
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.26022008061408997
Epoch: 3
Loss on hold-out set: 2.3951431490340322
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.24606913328170776
Epoch: 4
Loss on hold-out set: 2.3929684499524675
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.23742137849330902
Returned to Spot: Validation loss: 2.3929684499524675
----------------------------------------------
spotPython tuning: 2.3929684499524675 [----------] 4.00% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.4247119789088596, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'NAdam', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.397587686214807
Accuracy on hold-out set: 0.14622641509433962
MAPK value on hold-out data: 0.21776726841926575
Epoch: 2
Loss on hold-out set: 2.3969495004078127
Accuracy on hold-out set: 0.18396226415094338
MAPK value on hold-out data: 0.2838050127029419
Epoch: 3
Loss on hold-out set: 2.3959835700269014
Accuracy on hold-out set: 0.18396226415094338
MAPK value on hold-out data: 0.2845911383628845
Epoch: 4
Loss on hold-out set: 2.3948317248866244
Accuracy on hold-out set: 0.19811320754716982
MAPK value on hold-out data: 0.3254716396331787
Returned to Spot: Validation loss: 2.3948317248866244
----------------------------------------------
spotPython tuning: 2.3929684499524675 [----------] 4.63% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.32462859814607764, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 8, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.397664173594061
Accuracy on hold-out set: 0.07075471698113207
MAPK value on hold-out data: 0.14150945842266083
Epoch: 2
Loss on hold-out set: 2.3972570468794623
Accuracy on hold-out set: 0.08018867924528301
MAPK value on hold-out data: 0.16509436070919037
Epoch: 3
Loss on hold-out set: 2.3968688002172507
Accuracy on hold-out set: 0.08962264150943396
MAPK value on hold-out data: 0.18081757426261902
Epoch: 4
Loss on hold-out set: 2.3962746746135206
Accuracy on hold-out set: 0.08962264150943396
MAPK value on hold-out data: 0.17767290771007538
Returned to Spot: Validation loss: 2.3962746746135206
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#---------] 5.22% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.32599693008734476, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 8, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3974445203565202
Accuracy on hold-out set: 0.13679245283018868
MAPK value on hold-out data: 0.23191823065280914
Epoch: 2
Loss on hold-out set: 2.3971818303162196
Accuracy on hold-out set: 0.14622641509433962
MAPK value on hold-out data: 0.23899367451667786
Epoch: 3
Loss on hold-out set: 2.3969676764506214
Accuracy on hold-out set: 0.13679245283018868
MAPK value on hold-out data: 0.22562888264656067
Epoch: 4
Loss on hold-out set: 2.396499770992207
Accuracy on hold-out set: 0.14150943396226415
MAPK value on hold-out data: 0.2209119200706482
Returned to Spot: Validation loss: 2.396499770992207
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#---------] 5.81% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.3242702419988233, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 8, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.39715509144765
Accuracy on hold-out set: 0.10849056603773585
MAPK value on hold-out data: 0.22327043116092682
Epoch: 2
Loss on hold-out set: 2.396960328210075
Accuracy on hold-out set: 0.1179245283018868
MAPK value on hold-out data: 0.2342766970396042
Epoch: 3
Loss on hold-out set: 2.396492364271632
Accuracy on hold-out set: 0.1179245283018868
MAPK value on hold-out data: 0.2437106817960739
Epoch: 4
Loss on hold-out set: 2.3958631056659625
Accuracy on hold-out set: 0.1179245283018868
MAPK value on hold-out data: 0.22012577950954437
Returned to Spot: Validation loss: 2.3958631056659625
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#---------] 6.40% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.3223444892762921, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 8, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3976305835651903
Accuracy on hold-out set: 0.08490566037735849
MAPK value on hold-out data: 0.17295600473880768
Epoch: 2
Loss on hold-out set: 2.397296961748375
Accuracy on hold-out set: 0.09433962264150944
MAPK value on hold-out data: 0.1878930926322937
Epoch: 3
Loss on hold-out set: 2.396916153295985
Accuracy on hold-out set: 0.12264150943396226
MAPK value on hold-out data: 0.21855342388153076
Epoch: 4
Loss on hold-out set: 2.3963305477826102
Accuracy on hold-out set: 0.1320754716981132
MAPK value on hold-out data: 0.2342766970396042
Returned to Spot: Validation loss: 2.3963305477826102
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#---------] 6.98% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.3214657479149934, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 8, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.397742674035846
Accuracy on hold-out set: 0.1179245283018868
MAPK value on hold-out data: 0.17688676714897156
Epoch: 2
Loss on hold-out set: 2.3974522622126453
Accuracy on hold-out set: 0.10849056603773585
MAPK value on hold-out data: 0.18867920339107513
Epoch: 3
Loss on hold-out set: 2.397141024751483
Accuracy on hold-out set: 0.11320754716981132
MAPK value on hold-out data: 0.18396224081516266
Epoch: 4
Loss on hold-out set: 2.3966640144024254
Accuracy on hold-out set: 0.1179245283018868
MAPK value on hold-out data: 0.18474841117858887
Returned to Spot: Validation loss: 2.3966640144024254
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#---------] 7.58% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.36069547731509854, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 8, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.397711011598695
Accuracy on hold-out set: 0.09905660377358491
MAPK value on hold-out data: 0.1886792629957199
Epoch: 2
Loss on hold-out set: 2.3973531678037823
Accuracy on hold-out set: 0.14150943396226415
MAPK value on hold-out data: 0.21226412057876587
Epoch: 3
Loss on hold-out set: 2.3971096907021865
Accuracy on hold-out set: 0.1509433962264151
MAPK value on hold-out data: 0.220911905169487
Epoch: 4
Loss on hold-out set: 2.3965875067800844
Accuracy on hold-out set: 0.14150943396226415
MAPK value on hold-out data: 0.21698109805583954
Returned to Spot: Validation loss: 2.3965875067800844
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#---------] 8.17% 

config: {'_L0': 6112, 'l1': 4096, 'dropout_prob': 0.324974698584555, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'NAdam', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.39773472749962
Accuracy on hold-out set: 0.15566037735849056
MAPK value on hold-out data: 0.20597483217716217
Epoch: 2
Loss on hold-out set: 2.397241662133415
Accuracy on hold-out set: 0.16981132075471697
MAPK value on hold-out data: 0.26808175444602966
Epoch: 3
Loss on hold-out set: 2.3966532743202067
Accuracy on hold-out set: 0.18396226415094338
MAPK value on hold-out data: 0.26572322845458984
Epoch: 4
Loss on hold-out set: 2.395737108194603
Accuracy on hold-out set: 0.17452830188679244
MAPK value on hold-out data: 0.2751571834087372
Returned to Spot: Validation loss: 2.395737108194603
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#---------] 8.48% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.2942730989059993, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 8, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3975748116115354
Accuracy on hold-out set: 0.1320754716981132
MAPK value on hold-out data: 0.19811317324638367
Epoch: 2
Loss on hold-out set: 2.397274003838593
Accuracy on hold-out set: 0.12264150943396226
MAPK value on hold-out data: 0.2075471729040146
Epoch: 3
Loss on hold-out set: 2.396703106052471
Accuracy on hold-out set: 0.1320754716981132
MAPK value on hold-out data: 0.2216981202363968
Epoch: 4
Loss on hold-out set: 2.3960378597367487
Accuracy on hold-out set: 0.13679245283018868
MAPK value on hold-out data: 0.23742137849330902
Returned to Spot: Validation loss: 2.3960378597367487
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#---------] 9.14% 

config: {'_L0': 6112, 'l1': 4096, 'dropout_prob': 0.34099854145374997, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'NAdam', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3975734485770173
Accuracy on hold-out set: 0.09905660377358491
MAPK value on hold-out data: 0.16116352379322052
Epoch: 2
Loss on hold-out set: 2.3972476001055734
Accuracy on hold-out set: 0.09905660377358491
MAPK value on hold-out data: 0.19025152921676636
Epoch: 3
Loss on hold-out set: 2.3968195982699125
Accuracy on hold-out set: 0.12735849056603774
MAPK value on hold-out data: 0.22955970466136932
Epoch: 4
Loss on hold-out set: 2.3961308497302936
Accuracy on hold-out set: 0.11320754716981132
MAPK value on hold-out data: 0.24371062219142914
Returned to Spot: Validation loss: 2.3961308497302936
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#---------] 9.45% 

config: {'_L0': 6112, 'l1': 4096, 'dropout_prob': 0.32924653247743996, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'NAdam', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.397743483759322
Accuracy on hold-out set: 0.08490566037735849
MAPK value on hold-out data: 0.15487422049045563
Epoch: 2
Loss on hold-out set: 2.3973870299897104
Accuracy on hold-out set: 0.0660377358490566
MAPK value on hold-out data: 0.1250000149011612
Epoch: 3
Loss on hold-out set: 2.396953027203398
Accuracy on hold-out set: 0.07547169811320754
MAPK value on hold-out data: 0.13757863640785217
Epoch: 4
Loss on hold-out set: 2.39629324652114
Accuracy on hold-out set: 0.08490566037735849
MAPK value on hold-out data: 0.15723271667957306
Returned to Spot: Validation loss: 2.39629324652114
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#---------] 9.76% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.24690474183735678, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 8, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3975365994111546
Accuracy on hold-out set: 0.14622641509433962
MAPK value on hold-out data: 0.20361633598804474
Epoch: 2
Loss on hold-out set: 2.3969897144245653
Accuracy on hold-out set: 0.18396226415094338
MAPK value on hold-out data: 0.276729553937912
Epoch: 3
Loss on hold-out set: 2.396380348025628
Accuracy on hold-out set: 0.21226415094339623
MAPK value on hold-out data: 0.32075467705726624
Epoch: 4
Loss on hold-out set: 2.3956491002496683
Accuracy on hold-out set: 0.25
MAPK value on hold-out data: 0.3561320900917053
Returned to Spot: Validation loss: 2.3956491002496683
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#---------] 10.36% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.1428890825699365, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.397339496972426
Accuracy on hold-out set: 0.1320754716981132
MAPK value on hold-out data: 0.2224842756986618
Epoch: 2
Loss on hold-out set: 2.3968037029482283
Accuracy on hold-out set: 0.1179245283018868
MAPK value on hold-out data: 0.21226413547992706
Epoch: 3
Loss on hold-out set: 2.3961014860081224
Accuracy on hold-out set: 0.11320754716981132
MAPK value on hold-out data: 0.207547128200531
Epoch: 4
Loss on hold-out set: 2.3952107991812364
Accuracy on hold-out set: 0.11320754716981132
MAPK value on hold-out data: 0.21933957934379578
Returned to Spot: Validation loss: 2.3952107991812364
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#---------] 10.96% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.06014422563912306, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.397465206542105
Accuracy on hold-out set: 0.14622641509433962
MAPK value on hold-out data: 0.24449680745601654
Epoch: 2
Loss on hold-out set: 2.3967751984326346
Accuracy on hold-out set: 0.1792452830188679
MAPK value on hold-out data: 0.2704402506351471
Epoch: 3
Loss on hold-out set: 2.395953540532094
Accuracy on hold-out set: 0.18867924528301888
MAPK value on hold-out data: 0.2704402506351471
Epoch: 4
Loss on hold-out set: 2.39482622326545
Accuracy on hold-out set: 0.19811320754716982
MAPK value on hold-out data: 0.28773581981658936
Returned to Spot: Validation loss: 2.39482622326545
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#---------] 11.58% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.002839399566617521, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.397558997262199
Accuracy on hold-out set: 0.09905660377358491
MAPK value on hold-out data: 0.20676100254058838
Epoch: 2
Loss on hold-out set: 2.396918254078559
Accuracy on hold-out set: 0.12735849056603774
MAPK value on hold-out data: 0.20676100254058838
Epoch: 3
Loss on hold-out set: 2.3960215595533265
Accuracy on hold-out set: 0.1650943396226415
MAPK value on hold-out data: 0.2358490526676178
Epoch: 4
Loss on hold-out set: 2.3946717212784967
Accuracy on hold-out set: 0.18396226415094338
MAPK value on hold-out data: 0.24842765927314758
Returned to Spot: Validation loss: 2.3946717212784967
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#---------] 12.20% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.0, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3971944575039847
Accuracy on hold-out set: 0.1509433962264151
MAPK value on hold-out data: 0.24606916308403015
Epoch: 2
Loss on hold-out set: 2.3964820560419335
Accuracy on hold-out set: 0.1650943396226415
MAPK value on hold-out data: 0.27358487248420715
Epoch: 3
Loss on hold-out set: 2.395509659119372
Accuracy on hold-out set: 0.19811320754716982
MAPK value on hold-out data: 0.31053459644317627
Epoch: 4
Loss on hold-out set: 2.3940739564175875
Accuracy on hold-out set: 0.24056603773584906
MAPK value on hold-out data: 0.3388364613056183
Returned to Spot: Validation loss: 2.3940739564175875
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#---------] 12.85% 

config: {'_L0': 6112, 'l1': 1024, 'dropout_prob': 0.2490529292398342, 'lr_mult': 0.001, 'batch_size': 8, 'epochs': 4, 'k_folds': 1, 'patience': 32, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3981448897609003
Accuracy on hold-out set: 0.08962264150943396
MAPK value on hold-out data: 0.15509259700775146
Epoch: 2
Loss on hold-out set: 2.398100702850907
Accuracy on hold-out set: 0.08962264150943396
MAPK value on hold-out data: 0.15432099997997284
Epoch: 3
Loss on hold-out set: 2.398112968162254
Accuracy on hold-out set: 0.08490566037735849
MAPK value on hold-out data: 0.15123459696769714
Epoch: 4
Loss on hold-out set: 2.397998677359687
Accuracy on hold-out set: 0.09905660377358491
MAPK value on hold-out data: 0.16280867159366608
Returned to Spot: Validation loss: 2.397998677359687
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#---------] 12.93% 

config: {'_L0': 6112, 'l1': 4096, 'dropout_prob': 0.6899725481860932, 'lr_mult': 0.001, 'batch_size': 4, 'epochs': 4, 'k_folds': 1, 'patience': 8, 'optimizer': 'NAdam', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.398027793416437
Accuracy on hold-out set: 0.07547169811320754
MAPK value on hold-out data: 0.1525157392024994
Epoch: 2
Loss on hold-out set: 2.397770458797239
Accuracy on hold-out set: 0.13679245283018868
MAPK value on hold-out data: 0.1949685662984848
Epoch: 3
Loss on hold-out set: 2.3979763669787713
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.18396228551864624
Epoch: 4
Loss on hold-out set: 2.3977513943078383
Accuracy on hold-out set: 0.10849056603773585
MAPK value on hold-out data: 0.1878930777311325
Returned to Spot: Validation loss: 2.3977513943078383
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#---------] 13.11% 

config: {'_L0': 6112, 'l1': 128, 'dropout_prob': 0.8754395784486567, 'lr_mult': 0.001, 'batch_size': 4, 'epochs': 4, 'k_folds': 1, 'patience': 8, 'optimizer': 'Adamax', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3969692689067914
Accuracy on hold-out set: 0.12264150943396226
MAPK value on hold-out data: 0.21933962404727936
Epoch: 2
Loss on hold-out set: 2.397054969139819
Accuracy on hold-out set: 0.12264150943396226
MAPK value on hold-out data: 0.2146226018667221
Epoch: 3
Loss on hold-out set: 2.397040268160262
Accuracy on hold-out set: 0.12264150943396226
MAPK value on hold-out data: 0.21147798001766205
Epoch: 4
Loss on hold-out set: 2.396929124616227
Accuracy on hold-out set: 0.12264150943396226
MAPK value on hold-out data: 0.21069180965423584
Returned to Spot: Validation loss: 2.396929124616227
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#---------] 13.23% 

config: {'_L0': 6112, 'l1': 4096, 'dropout_prob': 0.5800401863861032, 'lr_mult': 0.001, 'batch_size': 8, 'epochs': 4, 'k_folds': 1, 'patience': 32, 'optimizer': 'Adamax', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3977003627353244
Accuracy on hold-out set: 0.08490566037735849
MAPK value on hold-out data: 0.15200617909431458
Epoch: 2
Loss on hold-out set: 2.397554874420166
Accuracy on hold-out set: 0.09905660377358491
MAPK value on hold-out data: 0.18441356718540192
Epoch: 3
Loss on hold-out set: 2.3976140287187366
Accuracy on hold-out set: 0.10849056603773585
MAPK value on hold-out data: 0.18827161192893982
Epoch: 4
Loss on hold-out set: 2.3975938426123724
Accuracy on hold-out set: 0.11320754716981132
MAPK value on hold-out data: 0.19290123879909515
Returned to Spot: Validation loss: 2.3975938426123724
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#---------] 13.33% 

config: {'_L0': 6112, 'l1': 1024, 'dropout_prob': 0.38034667719743903, 'lr_mult': 0.001, 'batch_size': 4, 'epochs': 4, 'k_folds': 1, 'patience': 4, 'optimizer': 'Adam', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3984579320223824
Accuracy on hold-out set: 0.07075471698113207
MAPK value on hold-out data: 0.1312893182039261
Epoch: 2
Loss on hold-out set: 2.398484320010779
Accuracy on hold-out set: 0.08018867924528301
MAPK value on hold-out data: 0.13443398475646973
Epoch: 3
Loss on hold-out set: 2.3982963202134617
Accuracy on hold-out set: 0.07075471698113207
MAPK value on hold-out data: 0.1320754736661911
Epoch: 4
Loss on hold-out set: 2.3983660329062984
Accuracy on hold-out set: 0.05660377358490566
MAPK value on hold-out data: 0.11635220795869827
Returned to Spot: Validation loss: 2.3983660329062984
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#---------] 13.44% 

config: {'_L0': 6112, 'l1': 256, 'dropout_prob': 0.7475409498559539, 'lr_mult': 0.001, 'batch_size': 4, 'epochs': 4, 'k_folds': 1, 'patience': 32, 'optimizer': 'Adam', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.398487680363205
Accuracy on hold-out set: 0.10849056603773585
MAPK value on hold-out data: 0.16194969415664673
Epoch: 2
Loss on hold-out set: 2.3984041348943173
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.14779874682426453
Epoch: 3
Loss on hold-out set: 2.3982887897851333
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.15723271667957306
Epoch: 4
Loss on hold-out set: 2.3984733887438505
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.15566037595272064
Returned to Spot: Validation loss: 2.3984733887438505
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#---------] 13.55% 

config: {'_L0': 6112, 'l1': 4096, 'dropout_prob': 0.31544842331355805, 'lr_mult': 0.001, 'batch_size': 4, 'epochs': 4, 'k_folds': 1, 'patience': 32, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3981406733674824
Accuracy on hold-out set: 0.08490566037735849
MAPK value on hold-out data: 0.14308178424835205
Epoch: 2
Loss on hold-out set: 2.3980641095143445
Accuracy on hold-out set: 0.08962264150943396
MAPK value on hold-out data: 0.1454402655363083
Epoch: 3
Loss on hold-out set: 2.397928057976489
Accuracy on hold-out set: 0.06132075471698113
MAPK value on hold-out data: 0.1328616589307785
Epoch: 4
Loss on hold-out set: 2.3977640035017482
Accuracy on hold-out set: 0.0660377358490566
MAPK value on hold-out data: 0.13679246604442596
Returned to Spot: Validation loss: 2.3977640035017482
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#---------] 13.71% 

config: {'_L0': 6112, 'l1': 2048, 'dropout_prob': 0.8650619774359263, 'lr_mult': 0.001, 'batch_size': 4, 'epochs': 4, 'k_folds': 1, 'patience': 8, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3975758777474456
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.18317607045173645
Epoch: 2
Loss on hold-out set: 2.3974072843227745
Accuracy on hold-out set: 0.08018867924528301
MAPK value on hold-out data: 0.18474844098091125
Epoch: 3
Loss on hold-out set: 2.3976868953344956
Accuracy on hold-out set: 0.08490566037735849
MAPK value on hold-out data: 0.19025155901908875
Epoch: 4
Loss on hold-out set: 2.3976050727772265
Accuracy on hold-out set: 0.12735849056603774
MAPK value on hold-out data: 0.19889935851097107
Returned to Spot: Validation loss: 2.3976050727772265
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#---------] 13.84% 

config: {'_L0': 6112, 'l1': 1024, 'dropout_prob': 0.2139388439945934, 'lr_mult': 0.001, 'batch_size': 8, 'epochs': 4, 'k_folds': 1, 'patience': 8, 'optimizer': 'Adamax', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.397774678689462
Accuracy on hold-out set: 0.09433962264150944
MAPK value on hold-out data: 0.18287035822868347
Epoch: 2
Loss on hold-out set: 2.397818609520241
Accuracy on hold-out set: 0.08962264150943396
MAPK value on hold-out data: 0.1736111044883728
Epoch: 3
Loss on hold-out set: 2.397680194289596
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.19675922393798828
Epoch: 4
Loss on hold-out set: 2.3976932013476335
Accuracy on hold-out set: 0.09433962264150944
MAPK value on hold-out data: 0.19058643281459808
Returned to Spot: Validation loss: 2.3976932013476335
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#---------] 13.92% 

config: {'_L0': 6112, 'l1': 128, 'dropout_prob': 0.03738935860756178, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 4, 'optimizer': 'Adamax', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.397186130847571
Accuracy on hold-out set: 0.1320754716981132
MAPK value on hold-out data: 0.1941823810338974
Epoch: 2
Loss on hold-out set: 2.3971957845507927
Accuracy on hold-out set: 0.1320754716981132
MAPK value on hold-out data: 0.19575469195842743
Epoch: 3
Loss on hold-out set: 2.3971349540746436
Accuracy on hold-out set: 0.1320754716981132
MAPK value on hold-out data: 0.19732701778411865
Epoch: 4
Loss on hold-out set: 2.3971832585784623
Accuracy on hold-out set: 0.1320754716981132
MAPK value on hold-out data: 0.19575469195842743
Returned to Spot: Validation loss: 2.3971832585784623
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#---------] 14.09% 

config: {'_L0': 6112, 'l1': 4096, 'dropout_prob': 0.21688592886999503, 'lr_mult': 0.001, 'batch_size': 4, 'epochs': 4, 'k_folds': 1, 'patience': 32, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3980822743109935
Accuracy on hold-out set: 0.0660377358490566
MAPK value on hold-out data: 0.12421383708715439
Epoch: 2
Loss on hold-out set: 2.3979672440942728
Accuracy on hold-out set: 0.0660377358490566
MAPK value on hold-out data: 0.11556603759527206
Epoch: 3
Loss on hold-out set: 2.3977572917938232
Accuracy on hold-out set: 0.05660377358490566
MAPK value on hold-out data: 0.12735848128795624
Epoch: 4
Loss on hold-out set: 2.3976237369033524
Accuracy on hold-out set: 0.05660377358490566
MAPK value on hold-out data: 0.11871069669723511
Returned to Spot: Validation loss: 2.3976237369033524
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#---------] 14.27% 

config: {'_L0': 6112, 'l1': 4096, 'dropout_prob': 0.06922100003001834, 'lr_mult': 0.001, 'batch_size': 4, 'epochs': 4, 'k_folds': 1, 'patience': 32, 'optimizer': 'Adamax', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3981575156157873
Accuracy on hold-out set: 0.08018867924528301
MAPK value on hold-out data: 0.13364781439304352
Epoch: 2
Loss on hold-out set: 2.3980562776889442
Accuracy on hold-out set: 0.07075471698113207
MAPK value on hold-out data: 0.12814466655254364
Epoch: 3
Loss on hold-out set: 2.397922560853778
Accuracy on hold-out set: 0.06132075471698113
MAPK value on hold-out data: 0.12735849618911743
Epoch: 4
Loss on hold-out set: 2.3978338376531063
Accuracy on hold-out set: 0.07075471698113207
MAPK value on hold-out data: 0.13600629568099976
Returned to Spot: Validation loss: 2.3978338376531063
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#---------] 14.44% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.444431018534004, 'lr_mult': 0.001, 'batch_size': 8, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'Adamax', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3978110949198403
Accuracy on hold-out set: 0.1179245283018868
MAPK value on hold-out data: 0.20447532832622528
Epoch: 2
Loss on hold-out set: 2.3976747813048185
Accuracy on hold-out set: 0.12735849056603774
MAPK value on hold-out data: 0.21604937314987183
Epoch: 3
Loss on hold-out set: 2.3975875201048673
Accuracy on hold-out set: 0.1179245283018868
MAPK value on hold-out data: 0.2260802537202835
Epoch: 4
Loss on hold-out set: 2.3975525343859636
Accuracy on hold-out set: 0.1320754716981132
MAPK value on hold-out data: 0.23148149251937866
Returned to Spot: Validation loss: 2.3975525343859636
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#---------] 14.64% 

config: {'_L0': 6112, 'l1': 128, 'dropout_prob': 0.15079665343745283, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'NAdam', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3984699856560185
Accuracy on hold-out set: 0.08018867924528301
MAPK value on hold-out data: 0.1533018797636032
Epoch: 2
Loss on hold-out set: 2.3983802817902475
Accuracy on hold-out set: 0.08018867924528301
MAPK value on hold-out data: 0.15487422049045563
Epoch: 3
Loss on hold-out set: 2.3983215835859193
Accuracy on hold-out set: 0.08018867924528301
MAPK value on hold-out data: 0.15487422049045563
Epoch: 4
Loss on hold-out set: 2.3982450242312447
Accuracy on hold-out set: 0.08018867924528301
MAPK value on hold-out data: 0.15566037595272064
Returned to Spot: Validation loss: 2.3982450242312447
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#---------] 14.81% 

config: {'_L0': 6112, 'l1': 512, 'dropout_prob': 0.42628859210850506, 'lr_mult': 0.001, 'batch_size': 16, 'epochs': 4, 'k_folds': 1, 'patience': 64, 'optimizer': 'NAdam', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3972210032599315
Accuracy on hold-out set: 0.07075471698113207
MAPK value on hold-out data: 0.1830357164144516
Epoch: 2
Loss on hold-out set: 2.397139344896589
Accuracy on hold-out set: 0.07075471698113207
MAPK value on hold-out data: 0.1793154776096344
Epoch: 3
Loss on hold-out set: 2.3971804550715854
Accuracy on hold-out set: 0.07075471698113207
MAPK value on hold-out data: 0.1815476268529892
Epoch: 4
Loss on hold-out set: 2.3972432613372803
Accuracy on hold-out set: 0.07075471698113207
MAPK value on hold-out data: 0.1800595223903656
Returned to Spot: Validation loss: 2.3972432613372803
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#---------] 14.87% 

config: {'_L0': 6112, 'l1': 128, 'dropout_prob': 0.7667268926608277, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 8, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3998689898904764
Accuracy on hold-out set: 0.08490566037735849
MAPK value on hold-out data: 0.15094341337680817
Epoch: 2
Loss on hold-out set: 2.399862804502811
Accuracy on hold-out set: 0.08490566037735849
MAPK value on hold-out data: 0.1470125913619995
Epoch: 3
Loss on hold-out set: 2.3998684388286664
Accuracy on hold-out set: 0.08490566037735849
MAPK value on hold-out data: 0.15801888704299927
Epoch: 4
Loss on hold-out set: 2.400035554507993
Accuracy on hold-out set: 0.07547169811320754
MAPK value on hold-out data: 0.1540880799293518
Returned to Spot: Validation loss: 2.400035554507993
----------------------------------------------
spotPython tuning: 2.3929684499524675 [##--------] 15.04% 

config: {'_L0': 6112, 'l1': 1024, 'dropout_prob': 0.6178629615502382, 'lr_mult': 0.001, 'batch_size': 4, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3975569572088853
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.17924527823925018
Epoch: 2
Loss on hold-out set: 2.3975137584614306
Accuracy on hold-out set: 0.12264150943396226
MAPK value on hold-out data: 0.19654086232185364
Epoch: 3
Loss on hold-out set: 2.3974704967354827
Accuracy on hold-out set: 0.12264150943396226
MAPK value on hold-out data: 0.19811317324638367
Epoch: 4
Loss on hold-out set: 2.3975366421465605
Accuracy on hold-out set: 0.11320754716981132
MAPK value on hold-out data: 0.1878930926322937
Returned to Spot: Validation loss: 2.3975366421465605
----------------------------------------------
spotPython tuning: 2.3929684499524675 [##--------] 15.15% 

config: {'_L0': 6112, 'l1': 256, 'dropout_prob': 0.8582308761741295, 'lr_mult': 0.001, 'batch_size': 16, 'epochs': 4, 'k_folds': 1, 'patience': 8, 'optimizer': 'Adamax', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.398734910147531
Accuracy on hold-out set: 0.0660377358490566
MAPK value on hold-out data: 0.1272321492433548
Epoch: 2
Loss on hold-out set: 2.3986475808279857
Accuracy on hold-out set: 0.07075471698113207
MAPK value on hold-out data: 0.1428571343421936
Epoch: 3
Loss on hold-out set: 2.398429887635367
Accuracy on hold-out set: 0.07075471698113207
MAPK value on hold-out data: 0.1287202388048172
Epoch: 4
Loss on hold-out set: 2.398554733821324
Accuracy on hold-out set: 0.07075471698113207
MAPK value on hold-out data: 0.131696417927742
Returned to Spot: Validation loss: 2.398554733821324
----------------------------------------------
spotPython tuning: 2.3929684499524675 [##--------] 15.20% 

config: {'_L0': 6112, 'l1': 128, 'dropout_prob': 0.05364579986179592, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 32, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.397782269513832
Accuracy on hold-out set: 0.0660377358490566
MAPK value on hold-out data: 0.16273586452007294
Epoch: 2
Loss on hold-out set: 2.3977355102323137
Accuracy on hold-out set: 0.0660377358490566
MAPK value on hold-out data: 0.16352201998233795
Epoch: 3
Loss on hold-out set: 2.397722784078346
Accuracy on hold-out set: 0.07075471698113207
MAPK value on hold-out data: 0.1666666716337204
Epoch: 4
Loss on hold-out set: 2.3976887532000273
Accuracy on hold-out set: 0.0660377358490566
MAPK value on hold-out data: 0.16430819034576416
Returned to Spot: Validation loss: 2.3976887532000273
----------------------------------------------
spotPython tuning: 2.3929684499524675 [##--------] 15.36% 

config: {'_L0': 6112, 'l1': 512, 'dropout_prob': 0.29870458505732056, 'lr_mult': 0.001, 'batch_size': 4, 'epochs': 4, 'k_folds': 1, 'patience': 32, 'optimizer': 'Adamax', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3977233508847795
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.19025154411792755
Epoch: 2
Loss on hold-out set: 2.3977043223830887
Accuracy on hold-out set: 0.11320754716981132
MAPK value on hold-out data: 0.19496853649616241
Epoch: 3
Loss on hold-out set: 2.3976647493974217
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.19968552887439728
Epoch: 4
Loss on hold-out set: 2.397697624170555
Accuracy on hold-out set: 0.09905660377358491
MAPK value on hold-out data: 0.19575469195842743
Returned to Spot: Validation loss: 2.397697624170555
----------------------------------------------
spotPython tuning: 2.3929684499524675 [##--------] 15.46% 

config: {'_L0': 6112, 'l1': 128, 'dropout_prob': 0.36981876582984996, 'lr_mult': 0.001, 'batch_size': 8, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'Adam', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.397944697627315
Accuracy on hold-out set: 0.12264150943396226
MAPK value on hold-out data: 0.18827159702777863
Epoch: 2
Loss on hold-out set: 2.3979869418674045
Accuracy on hold-out set: 0.14150943396226415
MAPK value on hold-out data: 0.1944444328546524
Epoch: 3
Loss on hold-out set: 2.397947214267872
Accuracy on hold-out set: 0.14150943396226415
MAPK value on hold-out data: 0.19521605968475342
Epoch: 4
Loss on hold-out set: 2.3979439205593533
Accuracy on hold-out set: 0.14150943396226415
MAPK value on hold-out data: 0.19675925374031067
Returned to Spot: Validation loss: 2.3979439205593533
----------------------------------------------
spotPython tuning: 2.3929684499524675 [##--------] 15.52% 

config: {'_L0': 6112, 'l1': 4096, 'dropout_prob': 0.6133830604749207, 'lr_mult': 0.001, 'batch_size': 8, 'epochs': 4, 'k_folds': 1, 'patience': 32, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3974273911228887
Accuracy on hold-out set: 0.13679245283018868
MAPK value on hold-out data: 0.20910492539405823
Epoch: 2
Loss on hold-out set: 2.397376855214437
Accuracy on hold-out set: 0.14150943396226415
MAPK value on hold-out data: 0.21990738809108734
Epoch: 3
Loss on hold-out set: 2.3973603071989835
Accuracy on hold-out set: 0.16037735849056603
MAPK value on hold-out data: 0.23765432834625244
Epoch: 4
Loss on hold-out set: 2.397356112798055
Accuracy on hold-out set: 0.16037735849056603
MAPK value on hold-out data: 0.24537032842636108
Returned to Spot: Validation loss: 2.397356112798055
----------------------------------------------
spotPython tuning: 2.3929684499524675 [##--------] 15.62% 

config: {'_L0': 6112, 'l1': 64, 'dropout_prob': 0.08763131596820395, 'lr_mult': 0.001, 'batch_size': 4, 'epochs': 4, 'k_folds': 1, 'patience': 64, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.400170231765171
Accuracy on hold-out set: 0.10849056603773585
MAPK value on hold-out data: 0.16352200508117676
Epoch: 2
Loss on hold-out set: 2.40012448688723
Accuracy on hold-out set: 0.10849056603773585
MAPK value on hold-out data: 0.16352200508117676
Epoch: 3
Loss on hold-out set: 2.400128126144409
Accuracy on hold-out set: 0.10849056603773585
MAPK value on hold-out data: 0.16352200508117676
Epoch: 4
Loss on hold-out set: 2.400109700436862
Accuracy on hold-out set: 0.10849056603773585
MAPK value on hold-out data: 0.16352200508117676
Returned to Spot: Validation loss: 2.400109700436862
----------------------------------------------
spotPython tuning: 2.3929684499524675 [##--------] 15.73% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.37470910898533233, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 8, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3975545010476744
Accuracy on hold-out set: 0.10849056603773585
MAPK value on hold-out data: 0.17924527823925018
Epoch: 2
Loss on hold-out set: 2.3973639011383057
Accuracy on hold-out set: 0.10849056603773585
MAPK value on hold-out data: 0.20440252125263214
Epoch: 3
Loss on hold-out set: 2.397053894006981
Accuracy on hold-out set: 0.10849056603773585
MAPK value on hold-out data: 0.2020440399646759
Epoch: 4
Loss on hold-out set: 2.3965317393248937
Accuracy on hold-out set: 0.10849056603773585
MAPK value on hold-out data: 0.18867924809455872
Returned to Spot: Validation loss: 2.3965317393248937
----------------------------------------------
spotPython tuning: 2.3929684499524675 [##--------] 16.34% 

config: {'_L0': 6112, 'l1': 2048, 'dropout_prob': 0.4169749444557389, 'lr_mult': 0.001, 'batch_size': 8, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'Adamax', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3982574056696007
Accuracy on hold-out set: 0.08018867924528301
MAPK value on hold-out data: 0.1427469104528427
Epoch: 2
Loss on hold-out set: 2.398382239871555
Accuracy on hold-out set: 0.08962264150943396
MAPK value on hold-out data: 0.1427469104528427
Epoch: 3
Loss on hold-out set: 2.398299570436831
Accuracy on hold-out set: 0.05660377358490566
MAPK value on hold-out data: 0.12731482088565826
Epoch: 4
Loss on hold-out set: 2.398282775172481
Accuracy on hold-out set: 0.09905660377358491
MAPK value on hold-out data: 0.15200616419315338
Returned to Spot: Validation loss: 2.398282775172481
----------------------------------------------
spotPython tuning: 2.3929684499524675 [##--------] 16.41% 

config: {'_L0': 6112, 'l1': 512, 'dropout_prob': 0.3287296306006683, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 8, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.398111986664106
Accuracy on hold-out set: 0.07075471698113207
MAPK value on hold-out data: 0.14386793971061707
Epoch: 2
Loss on hold-out set: 2.3980799508544632
Accuracy on hold-out set: 0.07075471698113207
MAPK value on hold-out data: 0.138364776968956
Epoch: 3
Loss on hold-out set: 2.3981317664092443
Accuracy on hold-out set: 0.07075471698113207
MAPK value on hold-out data: 0.1470125913619995
Epoch: 4
Loss on hold-out set: 2.3980268217482656
Accuracy on hold-out set: 0.07075471698113207
MAPK value on hold-out data: 0.13757863640785217
Returned to Spot: Validation loss: 2.3980268217482656
----------------------------------------------
spotPython tuning: 2.3929684499524675 [##--------] 16.64% 

config: {'_L0': 6112, 'l1': 2048, 'dropout_prob': 0.5849085505052557, 'lr_mult': 0.001, 'batch_size': 4, 'epochs': 4, 'k_folds': 1, 'patience': 32, 'optimizer': 'Adam', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3977596130011216
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.1666666567325592
Epoch: 2
Loss on hold-out set: 2.3978629067259014
Accuracy on hold-out set: 0.07547169811320754
MAPK value on hold-out data: 0.15094339847564697
Epoch: 3
Loss on hold-out set: 2.397666616259881
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.16116352379322052
Epoch: 4
Loss on hold-out set: 2.3975850096288718
Accuracy on hold-out set: 0.1179245283018868
MAPK value on hold-out data: 0.18553459644317627
Returned to Spot: Validation loss: 2.3975850096288718
----------------------------------------------
spotPython tuning: 2.3929684499524675 [##--------] 16.83% 

config: {'_L0': 6112, 'l1': 4096, 'dropout_prob': 0.25238335392942374, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.397877445760763
Accuracy on hold-out set: 0.09905660377358491
MAPK value on hold-out data: 0.1603773832321167
Epoch: 2
Loss on hold-out set: 2.3976203585570715
Accuracy on hold-out set: 0.14622641509433962
MAPK value on hold-out data: 0.22641503810882568
Epoch: 3
Loss on hold-out set: 2.39749150905969
Accuracy on hold-out set: 0.2028301886792453
MAPK value on hold-out data: 0.277515709400177
Epoch: 4
Loss on hold-out set: 2.3972808145127207
Accuracy on hold-out set: 0.22641509433962265
MAPK value on hold-out data: 0.30031445622444153
Returned to Spot: Validation loss: 2.3972808145127207
----------------------------------------------
spotPython tuning: 2.3929684499524675 [##--------] 17.21% 

config: {'_L0': 6112, 'l1': 1024, 'dropout_prob': 0.7734262466913263, 'lr_mult': 0.001, 'batch_size': 8, 'epochs': 4, 'k_folds': 1, 'patience': 8, 'optimizer': 'NAdam', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3975830519640886
Accuracy on hold-out set: 0.11320754716981132
MAPK value on hold-out data: 0.18364198505878448
Epoch: 2
Loss on hold-out set: 2.3975862485391124
Accuracy on hold-out set: 0.1179245283018868
MAPK value on hold-out data: 0.18904322385787964
Epoch: 3
Loss on hold-out set: 2.3975926593497947
Accuracy on hold-out set: 0.08490566037735849
MAPK value on hold-out data: 0.16435186564922333
Epoch: 4
Loss on hold-out set: 2.3975528787683555
Accuracy on hold-out set: 0.09905660377358491
MAPK value on hold-out data: 0.17206791043281555
Returned to Spot: Validation loss: 2.3975528787683555
----------------------------------------------
spotPython tuning: 2.3929684499524675 [##--------] 17.33% 

config: {'_L0': 6112, 'l1': 512, 'dropout_prob': 0.8251819533674807, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 8, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3978745847378136
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.20361633598804474
Epoch: 2
Loss on hold-out set: 2.3976536516873344
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.19654086232185364
Epoch: 3
Loss on hold-out set: 2.397562895181044
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.20676100254058838
Epoch: 4
Loss on hold-out set: 2.397957167535458
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.17059750854969025
Returned to Spot: Validation loss: 2.397957167535458
----------------------------------------------
spotPython tuning: 2.3929684499524675 [##--------] 17.60% 

config: {'_L0': 6112, 'l1': 64, 'dropout_prob': 0.18607520166859973, 'lr_mult': 0.001, 'batch_size': 16, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.397849542754037
Accuracy on hold-out set: 0.08962264150943396
MAPK value on hold-out data: 0.1711309403181076
Epoch: 2
Loss on hold-out set: 2.3977349996566772
Accuracy on hold-out set: 0.08962264150943396
MAPK value on hold-out data: 0.1726190596818924
Epoch: 3
Loss on hold-out set: 2.397806099482945
Accuracy on hold-out set: 0.08962264150943396
MAPK value on hold-out data: 0.1755952537059784
Epoch: 4
Loss on hold-out set: 2.3977251393454417
Accuracy on hold-out set: 0.08962264150943396
MAPK value on hold-out data: 0.1785714328289032
Returned to Spot: Validation loss: 2.3977251393454417
----------------------------------------------
spotPython tuning: 2.3929684499524675 [##--------] 17.72% 

config: {'_L0': 6112, 'l1': 256, 'dropout_prob': 0.3548477190453426, 'lr_mult': 0.001, 'batch_size': 16, 'epochs': 4, 'k_folds': 1, 'patience': 64, 'optimizer': 'Adam', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3985473939350674
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.1569940596818924
Epoch: 2
Loss on hold-out set: 2.398455704961504
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.1569940596818924
Epoch: 3
Loss on hold-out set: 2.398523654256548
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.1569940596818924
Epoch: 4
Loss on hold-out set: 2.398532135146005
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.1569940596818924
Returned to Spot: Validation loss: 2.398532135146005
----------------------------------------------
spotPython tuning: 2.3929684499524675 [##--------] 17.84% 

config: {'_L0': 6112, 'l1': 512, 'dropout_prob': 0.6733290439584552, 'lr_mult': 0.001, 'batch_size': 4, 'epochs': 4, 'k_folds': 1, 'patience': 32, 'optimizer': 'NAdam', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3980238122760125
Accuracy on hold-out set: 0.09433962264150944
MAPK value on hold-out data: 0.1525157392024994
Epoch: 2
Loss on hold-out set: 2.3980021071883866
Accuracy on hold-out set: 0.08018867924528301
MAPK value on hold-out data: 0.1454402655363083
Epoch: 3
Loss on hold-out set: 2.397999160694626
Accuracy on hold-out set: 0.09433962264150944
MAPK value on hold-out data: 0.15801887214183807
Epoch: 4
Loss on hold-out set: 2.39791457608061
Accuracy on hold-out set: 0.07547169811320754
MAPK value on hold-out data: 0.14072328805923462
Returned to Spot: Validation loss: 2.39791457608061
----------------------------------------------
spotPython tuning: 2.3929684499524675 [##--------] 18.01% 

config: {'_L0': 6112, 'l1': 128, 'dropout_prob': 0.12066571665703325, 'lr_mult': 0.001, 'batch_size': 8, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'Adamax', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.39888479091503
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.17283950746059418
Epoch: 2
Loss on hold-out set: 2.3988765787195274
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.17283950746059418
Epoch: 3
Loss on hold-out set: 2.3988715984203197
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.17438270151615143
Epoch: 4
Loss on hold-out set: 2.3988588209505433
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.17129629850387573
Returned to Spot: Validation loss: 2.3988588209505433
----------------------------------------------
spotPython tuning: 2.3929684499524675 [##--------] 18.15% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.824183031230104, 'lr_mult': 0.001, 'batch_size': 4, 'epochs': 4, 'k_folds': 1, 'patience': 32, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3983413183464193
Accuracy on hold-out set: 0.08962264150943396
MAPK value on hold-out data: 0.15251575410366058
Epoch: 2
Loss on hold-out set: 2.3981738090515137
Accuracy on hold-out set: 0.05660377358490566
MAPK value on hold-out data: 0.12185536324977875
Epoch: 3
Loss on hold-out set: 2.3980737767129576
Accuracy on hold-out set: 0.08490566037735849
MAPK value on hold-out data: 0.15094339847564697
Epoch: 4
Loss on hold-out set: 2.3977369722330346
Accuracy on hold-out set: 0.08962264150943396
MAPK value on hold-out data: 0.1588050127029419
Returned to Spot: Validation loss: 2.3977369722330346
----------------------------------------------
spotPython tuning: 2.3929684499524675 [##--------] 18.55% 

config: {'_L0': 6112, 'l1': 128, 'dropout_prob': 0.5246536981733558, 'lr_mult': 0.001, 'batch_size': 4, 'epochs': 4, 'k_folds': 1, 'patience': 4, 'optimizer': 'NAdam', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3967594470617906
Accuracy on hold-out set: 0.1179245283018868
MAPK value on hold-out data: 0.20676100254058838
Epoch: 2
Loss on hold-out set: 2.3967104093083798
Accuracy on hold-out set: 0.1179245283018868
MAPK value on hold-out data: 0.20676098763942719
Epoch: 3
Loss on hold-out set: 2.3966564007525175
Accuracy on hold-out set: 0.1179245283018868
MAPK value on hold-out data: 0.20833329856395721
Epoch: 4
Loss on hold-out set: 2.396720494864122
Accuracy on hold-out set: 0.1179245283018868
MAPK value on hold-out data: 0.2083333283662796
Returned to Spot: Validation loss: 2.396720494864122
----------------------------------------------
spotPython tuning: 2.3929684499524675 [##--------] 18.74% 

config: {'_L0': 6112, 'l1': 64, 'dropout_prob': 0.19311516116081714, 'lr_mult': 0.001, 'batch_size': 4, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'Adamax', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.400506662872602
Accuracy on hold-out set: 0.08018867924528301
MAPK value on hold-out data: 0.1462264358997345
Epoch: 2
Loss on hold-out set: 2.400490886760208
Accuracy on hold-out set: 0.08490566037735849
MAPK value on hold-out data: 0.14937108755111694
Epoch: 3
Loss on hold-out set: 2.400517562650285
Accuracy on hold-out set: 0.08018867924528301
MAPK value on hold-out data: 0.1470126062631607
Epoch: 4
Loss on hold-out set: 2.400505835155271
Accuracy on hold-out set: 0.08018867924528301
MAPK value on hold-out data: 0.14386793971061707
Returned to Spot: Validation loss: 2.400505835155271
----------------------------------------------
spotPython tuning: 2.3929684499524675 [##--------] 18.92% 

config: {'_L0': 6112, 'l1': 128, 'dropout_prob': 0.1605886907273366, 'lr_mult': 0.001, 'batch_size': 16, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3992779084614346
Accuracy on hold-out set: 0.07075471698113207
MAPK value on hold-out data: 0.1220238134264946
Epoch: 2
Loss on hold-out set: 2.3992346014295305
Accuracy on hold-out set: 0.07075471698113207
MAPK value on hold-out data: 0.1220238134264946
Epoch: 3
Loss on hold-out set: 2.399225047656468
Accuracy on hold-out set: 0.07075471698113207
MAPK value on hold-out data: 0.1220238134264946
Epoch: 4
Loss on hold-out set: 2.3991880416870117
Accuracy on hold-out set: 0.07075471698113207
MAPK value on hold-out data: 0.1220238134264946
Returned to Spot: Validation loss: 2.3991880416870117
----------------------------------------------
spotPython tuning: 2.3929684499524675 [##--------] 19.04% 

config: {'_L0': 6112, 'l1': 256, 'dropout_prob': 0.353749515623391, 'lr_mult': 0.001, 'batch_size': 16, 'epochs': 4, 'k_folds': 1, 'patience': 32, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.396805899483817
Accuracy on hold-out set: 0.1179245283018868
MAPK value on hold-out data: 0.2180059403181076
Epoch: 2
Loss on hold-out set: 2.3969105311802457
Accuracy on hold-out set: 0.1179245283018868
MAPK value on hold-out data: 0.2194940596818924
Epoch: 3
Loss on hold-out set: 2.3968722139086043
Accuracy on hold-out set: 0.1179245283018868
MAPK value on hold-out data: 0.2172619104385376
Epoch: 4
Loss on hold-out set: 2.396914839744568
Accuracy on hold-out set: 0.1179245283018868
MAPK value on hold-out data: 0.2202380746603012
Returned to Spot: Validation loss: 2.396914839744568
----------------------------------------------
spotPython tuning: 2.3929684499524675 [##--------] 19.23% 

config: {'_L0': 6112, 'l1': 256, 'dropout_prob': 0.30247202575852844, 'lr_mult': 0.001, 'batch_size': 16, 'epochs': 4, 'k_folds': 1, 'patience': 32, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3976214953831265
Accuracy on hold-out set: 0.0660377358490566
MAPK value on hold-out data: 0.1569940745830536
Epoch: 2
Loss on hold-out set: 2.3976509230477467
Accuracy on hold-out set: 0.09433962264150944
MAPK value on hold-out data: 0.1770833283662796
Epoch: 3
Loss on hold-out set: 2.3977381672177995
Accuracy on hold-out set: 0.06132075471698113
MAPK value on hold-out data: 0.1569940596818924
Epoch: 4
Loss on hold-out set: 2.3977231979370117
Accuracy on hold-out set: 0.05660377358490566
MAPK value on hold-out data: 0.1607142984867096
Returned to Spot: Validation loss: 2.3977231979370117
----------------------------------------------
spotPython tuning: 2.3929684499524675 [##--------] 19.40% 

config: {'_L0': 6112, 'l1': 64, 'dropout_prob': 0.41703000491938125, 'lr_mult': 0.001, 'batch_size': 8, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.398549344804552
Accuracy on hold-out set: 0.08490566037735849
MAPK value on hold-out data: 0.15740741789340973
Epoch: 2
Loss on hold-out set: 2.3985122398093894
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.1666666716337204
Epoch: 3
Loss on hold-out set: 2.3985288937886557
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.1666666716337204
Epoch: 4
Loss on hold-out set: 2.398548029087208
Accuracy on hold-out set: 0.09433962264150944
MAPK value on hold-out data: 0.16203702986240387
Returned to Spot: Validation loss: 2.398548029087208
----------------------------------------------
spotPython tuning: 2.3929684499524675 [##--------] 19.66% 

config: {'_L0': 6112, 'l1': 2048, 'dropout_prob': 0.5007096828230319, 'lr_mult': 0.001, 'batch_size': 4, 'epochs': 4, 'k_folds': 1, 'patience': 8, 'optimizer': 'Adamax', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3977668735216247
Accuracy on hold-out set: 0.10849056603773585
MAPK value on hold-out data: 0.16981130838394165
Epoch: 2
Loss on hold-out set: 2.3978631271506257
Accuracy on hold-out set: 0.09905660377358491
MAPK value on hold-out data: 0.15801888704299927
Epoch: 3
Loss on hold-out set: 2.397744781566116
Accuracy on hold-out set: 0.1179245283018868
MAPK value on hold-out data: 0.16902513802051544
Epoch: 4
Loss on hold-out set: 2.397724340546806
Accuracy on hold-out set: 0.11320754716981132
MAPK value on hold-out data: 0.16902516782283783
Returned to Spot: Validation loss: 2.397724340546806
----------------------------------------------
spotPython tuning: 2.3929684499524675 [##--------] 20.01% 

config: {'_L0': 6112, 'l1': 256, 'dropout_prob': 0.5901227741854255, 'lr_mult': 0.001, 'batch_size': 16, 'epochs': 4, 'k_folds': 1, 'patience': 32, 'optimizer': 'Adamax', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.397937365940639
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.1889881044626236
Epoch: 2
Loss on hold-out set: 2.3979208299091885
Accuracy on hold-out set: 0.11320754716981132
MAPK value on hold-out data: 0.2053571492433548
Epoch: 3
Loss on hold-out set: 2.397949916975839
Accuracy on hold-out set: 0.11320754716981132
MAPK value on hold-out data: 0.1919642835855484
Epoch: 4
Loss on hold-out set: 2.397943922451564
Accuracy on hold-out set: 0.11320754716981132
MAPK value on hold-out data: 0.1994047462940216
Returned to Spot: Validation loss: 2.397943922451564
----------------------------------------------
spotPython tuning: 2.3929684499524675 [##--------] 20.28% 

config: {'_L0': 6112, 'l1': 512, 'dropout_prob': 0.1692224028272108, 'lr_mult': 0.001, 'batch_size': 4, 'epochs': 4, 'k_folds': 1, 'patience': 32, 'optimizer': 'NAdam', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3982097382815377
Accuracy on hold-out set: 0.08962264150943396
MAPK value on hold-out data: 0.1399371176958084
Epoch: 2
Loss on hold-out set: 2.398194596452533
Accuracy on hold-out set: 0.05660377358490566
MAPK value on hold-out data: 0.12028302997350693
Epoch: 3
Loss on hold-out set: 2.3981220902137035
Accuracy on hold-out set: 0.08490566037735849
MAPK value on hold-out data: 0.13522014021873474
Epoch: 4
Loss on hold-out set: 2.3979895924622157
Accuracy on hold-out set: 0.0660377358490566
MAPK value on hold-out data: 0.1257861852645874
Returned to Spot: Validation loss: 2.3979895924622157
----------------------------------------------
spotPython tuning: 2.3929684499524675 [##--------] 20.56% 

config: {'_L0': 6112, 'l1': 128, 'dropout_prob': 0.6947457353404383, 'lr_mult': 0.001, 'batch_size': 4, 'epochs': 4, 'k_folds': 1, 'patience': 32, 'optimizer': 'Adamax', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3993853263135225
Accuracy on hold-out set: 0.06132075471698113
MAPK value on hold-out data: 0.14386793971061707
Epoch: 2
Loss on hold-out set: 2.3993851778642186
Accuracy on hold-out set: 0.06132075471698113
MAPK value on hold-out data: 0.14229561388492584
Epoch: 3
Loss on hold-out set: 2.399478997824327
Accuracy on hold-out set: 0.06132075471698113
MAPK value on hold-out data: 0.1454402655363083
Epoch: 4
Loss on hold-out set: 2.3994948099244318
Accuracy on hold-out set: 0.06132075471698113
MAPK value on hold-out data: 0.14386793971061707
Returned to Spot: Validation loss: 2.3994948099244318
----------------------------------------------
spotPython tuning: 2.3929684499524675 [##--------] 20.84% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.0014037572247290089, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3975030363730663
Accuracy on hold-out set: 0.14150943396226415
MAPK value on hold-out data: 0.2099056839942932
Epoch: 2
Loss on hold-out set: 2.3968961216368765
Accuracy on hold-out set: 0.13679245283018868
MAPK value on hold-out data: 0.23113200068473816
Epoch: 3
Loss on hold-out set: 2.396074317536264
Accuracy on hold-out set: 0.1179245283018868
MAPK value on hold-out data: 0.21933956444263458
Epoch: 4
Loss on hold-out set: 2.3949212353184537
Accuracy on hold-out set: 0.12735849056603774
MAPK value on hold-out data: 0.2334904819726944
Returned to Spot: Validation loss: 2.3949212353184537
----------------------------------------------
spotPython tuning: 2.3929684499524675 [##--------] 21.58% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.011001461500103234, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3975486777863413
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.16981133818626404
Epoch: 2
Loss on hold-out set: 2.397074688155696
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.19103772938251495
Epoch: 3
Loss on hold-out set: 2.396355392797938
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.20440249145030975
Epoch: 4
Loss on hold-out set: 2.3953544216335945
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.21776729822158813
Returned to Spot: Validation loss: 2.3953544216335945
----------------------------------------------
spotPython tuning: 2.3929684499524675 [##--------] 22.37% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.019776519596520287, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.397287071875806
Accuracy on hold-out set: 0.13679245283018868
MAPK value on hold-out data: 0.2004716843366623
Epoch: 2
Loss on hold-out set: 2.3966015982178024
Accuracy on hold-out set: 0.13679245283018868
MAPK value on hold-out data: 0.209119513630867
Epoch: 3
Loss on hold-out set: 2.3956827600047275
Accuracy on hold-out set: 0.14150943396226415
MAPK value on hold-out data: 0.23977984488010406
Epoch: 4
Loss on hold-out set: 2.3944050500977716
Accuracy on hold-out set: 0.16981132075471697
MAPK value on hold-out data: 0.26022011041641235
Returned to Spot: Validation loss: 2.3944050500977716
----------------------------------------------
spotPython tuning: 2.3929684499524675 [##--------] 23.11% 

config: {'_L0': 6112, 'l1': 2048, 'dropout_prob': 0.3745499929545773, 'lr_mult': 0.001, 'batch_size': 4, 'epochs': 4, 'k_folds': 1, 'patience': 64, 'optimizer': 'NAdam', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3974625209592424
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.19025155901908875
Epoch: 2
Loss on hold-out set: 2.3972605174442507
Accuracy on hold-out set: 0.08962264150943396
MAPK value on hold-out data: 0.18396227061748505
Epoch: 3
Loss on hold-out set: 2.397060124379284
Accuracy on hold-out set: 0.09433962264150944
MAPK value on hold-out data: 0.1863207370042801
Epoch: 4
Loss on hold-out set: 2.397071420021777
Accuracy on hold-out set: 0.09433962264150944
MAPK value on hold-out data: 0.18474844098091125
Returned to Spot: Validation loss: 2.397071420021777
----------------------------------------------
spotPython tuning: 2.3929684499524675 [##--------] 23.48% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.017448875113150207, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3974423655923807
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.2154088169336319
Epoch: 2
Loss on hold-out set: 2.396714248747196
Accuracy on hold-out set: 0.12735849056603774
MAPK value on hold-out data: 0.23034588992595673
Epoch: 3
Loss on hold-out set: 2.3957639977617085
Accuracy on hold-out set: 0.11320754716981132
MAPK value on hold-out data: 0.23977981507778168
Epoch: 4
Loss on hold-out set: 2.394420605785442
Accuracy on hold-out set: 0.16037735849056603
MAPK value on hold-out data: 0.26415085792541504
Returned to Spot: Validation loss: 2.394420605785442
----------------------------------------------
spotPython tuning: 2.3929684499524675 [##--------] 24.30% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.01665486398582384, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3970568270053505
Accuracy on hold-out set: 0.1320754716981132
MAPK value on hold-out data: 0.29481127858161926
Epoch: 2
Loss on hold-out set: 2.396394090832404
Accuracy on hold-out set: 0.13679245283018868
MAPK value on hold-out data: 0.31603771448135376
Epoch: 3
Loss on hold-out set: 2.3954651603158914
Accuracy on hold-out set: 0.16037735849056603
MAPK value on hold-out data: 0.32704395055770874
Epoch: 4
Loss on hold-out set: 2.3942137889142305
Accuracy on hold-out set: 0.1792452830188679
MAPK value on hold-out data: 0.3396226167678833
Returned to Spot: Validation loss: 2.3942137889142305
----------------------------------------------
spotPython tuning: 2.3929684499524675 [###-------] 25.06% 

config: {'_L0': 6112, 'l1': 512, 'dropout_prob': 0.6345094073291901, 'lr_mult': 0.001, 'batch_size': 8, 'epochs': 4, 'k_folds': 1, 'patience': 8, 'optimizer': 'Adamax', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.398092835037797
Accuracy on hold-out set: 0.08490566037735849
MAPK value on hold-out data: 0.1774691343307495
Epoch: 2
Loss on hold-out set: 2.3981528723681413
Accuracy on hold-out set: 0.08962264150943396
MAPK value on hold-out data: 0.16512347757816315
Epoch: 3
Loss on hold-out set: 2.398230482030798
Accuracy on hold-out set: 0.09433962264150944
MAPK value on hold-out data: 0.1736111044883728
Epoch: 4
Loss on hold-out set: 2.398093656257347
Accuracy on hold-out set: 0.09433962264150944
MAPK value on hold-out data: 0.1705246865749359
Returned to Spot: Validation loss: 2.398093656257347
----------------------------------------------
spotPython tuning: 2.3929684499524675 [###-------] 25.33% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.014552266103968559, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3973030031852
Accuracy on hold-out set: 0.12735849056603774
MAPK value on hold-out data: 0.22641505300998688
Epoch: 2
Loss on hold-out set: 2.396752953529358
Accuracy on hold-out set: 0.1320754716981132
MAPK value on hold-out data: 0.24606916308403015
Epoch: 3
Loss on hold-out set: 2.3959669549510165
Accuracy on hold-out set: 0.1179245283018868
MAPK value on hold-out data: 0.25235846638679504
Epoch: 4
Loss on hold-out set: 2.3948499796525486
Accuracy on hold-out set: 0.12735849056603774
MAPK value on hold-out data: 0.26493704319000244
Returned to Spot: Validation loss: 2.3948499796525486
----------------------------------------------
spotPython tuning: 2.3929684499524675 [###-------] 26.15% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.01605005668118086, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3973777901451543
Accuracy on hold-out set: 0.1792452830188679
MAPK value on hold-out data: 0.29402509331703186
Epoch: 2
Loss on hold-out set: 2.396694246328102
Accuracy on hold-out set: 0.21226415094339623
MAPK value on hold-out data: 0.3176100254058838
Epoch: 3
Loss on hold-out set: 2.395786903939157
Accuracy on hold-out set: 0.2169811320754717
MAPK value on hold-out data: 0.3231131434440613
Epoch: 4
Loss on hold-out set: 2.3944874754491843
Accuracy on hold-out set: 0.2028301886792453
MAPK value on hold-out data: 0.3176099956035614
Returned to Spot: Validation loss: 2.3944874754491843
----------------------------------------------
spotPython tuning: 2.3929684499524675 [###-------] 27.07% 

config: {'_L0': 6112, 'l1': 128, 'dropout_prob': 0.8586493837448784, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 8, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3981962473887317
Accuracy on hold-out set: 0.05660377358490566
MAPK value on hold-out data: 0.1391509771347046
Epoch: 2
Loss on hold-out set: 2.3979921093526877
Accuracy on hold-out set: 0.08962264150943396
MAPK value on hold-out data: 0.1666666567325592
Epoch: 3
Loss on hold-out set: 2.3980856616542026
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.16116350889205933
Epoch: 4
Loss on hold-out set: 2.39826914724314
Accuracy on hold-out set: 0.05188679245283019
MAPK value on hold-out data: 0.12657234072685242
Returned to Spot: Validation loss: 2.39826914724314
----------------------------------------------
spotPython tuning: 2.3929684499524675 [###-------] 27.44% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.0206156419301559, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.396957775331893
Accuracy on hold-out set: 0.13679245283018868
MAPK value on hold-out data: 0.21305030584335327
Epoch: 2
Loss on hold-out set: 2.396266660600338
Accuracy on hold-out set: 0.12264150943396226
MAPK value on hold-out data: 0.21383647620677948
Epoch: 3
Loss on hold-out set: 2.3953747142036006
Accuracy on hold-out set: 0.1179245283018868
MAPK value on hold-out data: 0.2154087871313095
Epoch: 4
Loss on hold-out set: 2.3941475602815734
Accuracy on hold-out set: 0.12735849056603774
MAPK value on hold-out data: 0.234276682138443
Returned to Spot: Validation loss: 2.3941475602815734
----------------------------------------------
spotPython tuning: 2.3929684499524675 [###-------] 28.24% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.022003112983107646, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3973145057570258
Accuracy on hold-out set: 0.10849056603773585
MAPK value on hold-out data: 0.18632076680660248
Epoch: 2
Loss on hold-out set: 2.39671790599823
Accuracy on hold-out set: 0.12735849056603774
MAPK value on hold-out data: 0.23742131888866425
Epoch: 3
Loss on hold-out set: 2.395874520517745
Accuracy on hold-out set: 0.12735849056603774
MAPK value on hold-out data: 0.2547169327735901
Epoch: 4
Loss on hold-out set: 2.394778213411007
Accuracy on hold-out set: 0.14150943396226415
MAPK value on hold-out data: 0.26022008061408997
Returned to Spot: Validation loss: 2.394778213411007
----------------------------------------------
spotPython tuning: 2.3929684499524675 [###-------] 29.10% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.021286297364420476, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3973607049798065
Accuracy on hold-out set: 0.10849056603773585
MAPK value on hold-out data: 0.2138364315032959
Epoch: 2
Loss on hold-out set: 2.396732991596438
Accuracy on hold-out set: 0.10849056603773585
MAPK value on hold-out data: 0.23742137849330902
Epoch: 3
Loss on hold-out set: 2.3957870321453743
Accuracy on hold-out set: 0.11320754716981132
MAPK value on hold-out data: 0.2633647620677948
Epoch: 4
Loss on hold-out set: 2.3944671311468446
Accuracy on hold-out set: 0.12735849056603774
MAPK value on hold-out data: 0.27830183506011963
Returned to Spot: Validation loss: 2.3944671311468446
----------------------------------------------
spotPython tuning: 2.3929684499524675 [###-------] 30.00% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.021784919218615924, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3976469512255685
Accuracy on hold-out set: 0.08490566037735849
MAPK value on hold-out data: 0.15487422049045563
Epoch: 2
Loss on hold-out set: 2.39709003241557
Accuracy on hold-out set: 0.08490566037735849
MAPK value on hold-out data: 0.15566037595272064
Epoch: 3
Loss on hold-out set: 2.396326038072694
Accuracy on hold-out set: 0.08490566037735849
MAPK value on hold-out data: 0.16037733852863312
Epoch: 4
Loss on hold-out set: 2.3953535331870026
Accuracy on hold-out set: 0.08490566037735849
MAPK value on hold-out data: 0.16902516782283783
Returned to Spot: Validation loss: 2.3953535331870026
----------------------------------------------
spotPython tuning: 2.3929684499524675 [###-------] 30.89% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.019262294536507896, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.397354445367489
Accuracy on hold-out set: 0.10849056603773585
MAPK value on hold-out data: 0.21147798001766205
Epoch: 2
Loss on hold-out set: 2.396685688000805
Accuracy on hold-out set: 0.15566037735849056
MAPK value on hold-out data: 0.2484276294708252
Epoch: 3
Loss on hold-out set: 2.3958260653153904
Accuracy on hold-out set: 0.17452830188679244
MAPK value on hold-out data: 0.2570754587650299
Epoch: 4
Loss on hold-out set: 2.3946308167475574
Accuracy on hold-out set: 0.1792452830188679
MAPK value on hold-out data: 0.27358484268188477
Returned to Spot: Validation loss: 2.3946308167475574
----------------------------------------------
spotPython tuning: 2.3929684499524675 [###-------] 31.80% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.019223362158809782, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3974154377883337
Accuracy on hold-out set: 0.09905660377358491
MAPK value on hold-out data: 0.17138366401195526
Epoch: 2
Loss on hold-out set: 2.3967823689838625
Accuracy on hold-out set: 0.1320754716981132
MAPK value on hold-out data: 0.24371066689491272
Epoch: 3
Loss on hold-out set: 2.395958954433225
Accuracy on hold-out set: 0.1509433962264151
MAPK value on hold-out data: 0.2539307773113251
Epoch: 4
Loss on hold-out set: 2.394877352804508
Accuracy on hold-out set: 0.1792452830188679
MAPK value on hold-out data: 0.3026729226112366
Returned to Spot: Validation loss: 2.394877352804508
----------------------------------------------
spotPython tuning: 2.3929684499524675 [###-------] 32.71% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.019143194887420834, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3973148139017932
Accuracy on hold-out set: 0.12264150943396226
MAPK value on hold-out data: 0.19339624047279358
Epoch: 2
Loss on hold-out set: 2.3967647147628495
Accuracy on hold-out set: 0.12735849056603774
MAPK value on hold-out data: 0.20597483217716217
Epoch: 3
Loss on hold-out set: 2.3960073961401886
Accuracy on hold-out set: 0.12735849056603774
MAPK value on hold-out data: 0.22877353429794312
Epoch: 4
Loss on hold-out set: 2.3949645487767346
Accuracy on hold-out set: 0.1320754716981132
MAPK value on hold-out data: 0.2507861256599426
Returned to Spot: Validation loss: 2.3949645487767346
----------------------------------------------
spotPython tuning: 2.3929684499524675 [###-------] 33.73% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.01909839506896736, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3973904510713973
Accuracy on hold-out set: 0.1179245283018868
MAPK value on hold-out data: 0.19968552887439728
Epoch: 2
Loss on hold-out set: 2.3966855283053414
Accuracy on hold-out set: 0.1179245283018868
MAPK value on hold-out data: 0.2146226167678833
Epoch: 3
Loss on hold-out set: 2.395765286571575
Accuracy on hold-out set: 0.11320754716981132
MAPK value on hold-out data: 0.22641509771347046
Epoch: 4
Loss on hold-out set: 2.3944662337033256
Accuracy on hold-out set: 0.11320754716981132
MAPK value on hold-out data: 0.23742131888866425
Returned to Spot: Validation loss: 2.3944662337033256
----------------------------------------------
spotPython tuning: 2.3929684499524675 [###-------] 34.68% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.019026654110531383, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.39742792327449
Accuracy on hold-out set: 0.1320754716981132
MAPK value on hold-out data: 0.2350628674030304
Epoch: 2
Loss on hold-out set: 2.3967117813398255
Accuracy on hold-out set: 0.20754716981132076
MAPK value on hold-out data: 0.3113207221031189
Epoch: 3
Loss on hold-out set: 2.3958202240602025
Accuracy on hold-out set: 0.25
MAPK value on hold-out data: 0.3702830672264099
Epoch: 4
Loss on hold-out set: 2.3945762931175953
Accuracy on hold-out set: 0.27358490566037735
MAPK value on hold-out data: 0.3954402506351471
Returned to Spot: Validation loss: 2.3945762931175953
----------------------------------------------
spotPython tuning: 2.3929684499524675 [####------] 35.70% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.01893677279602263, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.397363111657916
Accuracy on hold-out set: 0.1179245283018868
MAPK value on hold-out data: 0.1949685513973236
Epoch: 2
Loss on hold-out set: 2.3968414135699003
Accuracy on hold-out set: 0.10849056603773585
MAPK value on hold-out data: 0.21776728332042694
Epoch: 3
Loss on hold-out set: 2.3961071360786006
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.2295597344636917
Epoch: 4
Loss on hold-out set: 2.3950391895366163
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.23427671194076538
Returned to Spot: Validation loss: 2.3950391895366163
----------------------------------------------
spotPython tuning: 2.3929684499524675 [####------] 36.79% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.018974952153891105, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3971605773242013
Accuracy on hold-out set: 0.15566037735849056
MAPK value on hold-out data: 0.2421383559703827
Epoch: 2
Loss on hold-out set: 2.3965808333091014
Accuracy on hold-out set: 0.16981132075471697
MAPK value on hold-out data: 0.24685530364513397
Epoch: 3
Loss on hold-out set: 2.3957504663827285
Accuracy on hold-out set: 0.13679245283018868
MAPK value on hold-out data: 0.2484276443719864
Epoch: 4
Loss on hold-out set: 2.394490865041625
Accuracy on hold-out set: 0.12264150943396226
MAPK value on hold-out data: 0.2696540653705597
Returned to Spot: Validation loss: 2.394490865041625
----------------------------------------------
spotPython tuning: 2.3929684499524675 [####------] 37.79% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.018876208489202653, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.397069665620912
Accuracy on hold-out set: 0.11320754716981132
MAPK value on hold-out data: 0.21776722371578217
Epoch: 2
Loss on hold-out set: 2.3963418726651176
Accuracy on hold-out set: 0.12264150943396226
MAPK value on hold-out data: 0.23113200068473816
Epoch: 3
Loss on hold-out set: 2.3953668193997077
Accuracy on hold-out set: 0.14150943396226415
MAPK value on hold-out data: 0.2547169029712677
Epoch: 4
Loss on hold-out set: 2.3939907190934666
Accuracy on hold-out set: 0.16037735849056603
MAPK value on hold-out data: 0.2814464569091797
Returned to Spot: Validation loss: 2.3939907190934666
----------------------------------------------
spotPython tuning: 2.3929684499524675 [####------] 38.80% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.01899816026359838, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3972987453892545
Accuracy on hold-out set: 0.13679245283018868
MAPK value on hold-out data: 0.22484277188777924
Epoch: 2
Loss on hold-out set: 2.3966854608283854
Accuracy on hold-out set: 0.13679245283018868
MAPK value on hold-out data: 0.23742136359214783
Epoch: 3
Loss on hold-out set: 2.3958613962497353
Accuracy on hold-out set: 0.1320754716981132
MAPK value on hold-out data: 0.23820753395557404
Epoch: 4
Loss on hold-out set: 2.3947822350376056
Accuracy on hold-out set: 0.13679245283018868
MAPK value on hold-out data: 0.24606913328170776
Returned to Spot: Validation loss: 2.3947822350376056
----------------------------------------------
spotPython tuning: 2.3929684499524675 [####------] 39.72% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.0189389812039685, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.397113653848756
Accuracy on hold-out set: 0.15566037735849056
MAPK value on hold-out data: 0.26179239153862
Epoch: 2
Loss on hold-out set: 2.3964381937710746
Accuracy on hold-out set: 0.14150943396226415
MAPK value on hold-out data: 0.27279865741729736
Epoch: 3
Loss on hold-out set: 2.3955721427809515
Accuracy on hold-out set: 0.19339622641509435
MAPK value on hold-out data: 0.30817604064941406
Epoch: 4
Loss on hold-out set: 2.394370492899193
Accuracy on hold-out set: 0.19811320754716982
MAPK value on hold-out data: 0.31603774428367615
Returned to Spot: Validation loss: 2.394370492899193
----------------------------------------------
spotPython tuning: 2.3929684499524675 [####------] 40.65% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.018921664358582023, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3973889710768215
Accuracy on hold-out set: 0.1179245283018868
MAPK value on hold-out data: 0.19889938831329346
Epoch: 2
Loss on hold-out set: 2.396894526931475
Accuracy on hold-out set: 0.07075471698113207
MAPK value on hold-out data: 0.19889932870864868
Epoch: 3
Loss on hold-out set: 2.3962306728902854
Accuracy on hold-out set: 0.0660377358490566
MAPK value on hold-out data: 0.1933962106704712
Epoch: 4
Loss on hold-out set: 2.395408828303499
Accuracy on hold-out set: 0.0660377358490566
MAPK value on hold-out data: 0.1878930777311325
Returned to Spot: Validation loss: 2.395408828303499
----------------------------------------------
spotPython tuning: 2.3929684499524675 [####------] 41.59% 

config: {'_L0': 6112, 'l1': 512, 'dropout_prob': 0.11654668035810149, 'lr_mult': 0.001, 'batch_size': 16, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.396795289857047
Accuracy on hold-out set: 0.09905660377358491
MAPK value on hold-out data: 0.2120535522699356
Epoch: 2
Loss on hold-out set: 2.3967428718294417
Accuracy on hold-out set: 0.09905660377358491
MAPK value on hold-out data: 0.212797611951828
Epoch: 3
Loss on hold-out set: 2.3967396872384206
Accuracy on hold-out set: 0.09905660377358491
MAPK value on hold-out data: 0.212797611951828
Epoch: 4
Loss on hold-out set: 2.396719370569502
Accuracy on hold-out set: 0.09905660377358491
MAPK value on hold-out data: 0.2113095223903656
Returned to Spot: Validation loss: 2.396719370569502
----------------------------------------------
spotPython tuning: 2.3929684499524675 [####------] 41.96% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.01879075296171935, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.397654225241463
Accuracy on hold-out set: 0.06132075471698113
MAPK value on hold-out data: 0.13757863640785217
Epoch: 2
Loss on hold-out set: 2.39698836713467
Accuracy on hold-out set: 0.08018867924528301
MAPK value on hold-out data: 0.15566039085388184
Epoch: 3
Loss on hold-out set: 2.3961330877160125
Accuracy on hold-out set: 0.09433962264150944
MAPK value on hold-out data: 0.18553456664085388
Epoch: 4
Loss on hold-out set: 2.3950138699333623
Accuracy on hold-out set: 0.1509433962264151
MAPK value on hold-out data: 0.23820750415325165
Returned to Spot: Validation loss: 2.3950138699333623
----------------------------------------------
spotPython tuning: 2.3929684499524675 [####------] 43.04% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.01895447987190486, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.396923047191692
Accuracy on hold-out set: 0.14150943396226415
MAPK value on hold-out data: 0.2578616142272949
Epoch: 2
Loss on hold-out set: 2.3962565637984365
Accuracy on hold-out set: 0.14622641509433962
MAPK value on hold-out data: 0.27830180525779724
Epoch: 3
Loss on hold-out set: 2.3953328987337508
Accuracy on hold-out set: 0.13679245283018868
MAPK value on hold-out data: 0.27358484268188477
Epoch: 4
Loss on hold-out set: 2.3940581533144103
Accuracy on hold-out set: 0.15566037735849056
MAPK value on hold-out data: 0.2672955393791199
Returned to Spot: Validation loss: 2.3940581533144103
----------------------------------------------
spotPython tuning: 2.3929684499524675 [####------] 44.04% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.018789311020543393, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3974172146815174
Accuracy on hold-out set: 0.1509433962264151
MAPK value on hold-out data: 0.20597481727600098
Epoch: 2
Loss on hold-out set: 2.396836951093854
Accuracy on hold-out set: 0.12735849056603774
MAPK value on hold-out data: 0.207547128200531
Epoch: 3
Loss on hold-out set: 2.396080349976162
Accuracy on hold-out set: 0.1792452830188679
MAPK value on hold-out data: 0.2759433686733246
Epoch: 4
Loss on hold-out set: 2.395054967898243
Accuracy on hold-out set: 0.27358490566037735
MAPK value on hold-out data: 0.3529874086380005
Returned to Spot: Validation loss: 2.395054967898243
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#####-----] 45.06% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.0189812233526638, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.397338201414864
Accuracy on hold-out set: 0.12735849056603774
MAPK value on hold-out data: 0.2201257348060608
Epoch: 2
Loss on hold-out set: 2.396705933336942
Accuracy on hold-out set: 0.15566037735849056
MAPK value on hold-out data: 0.2539307773113251
Epoch: 3
Loss on hold-out set: 2.3959069139552565
Accuracy on hold-out set: 0.15566037735849056
MAPK value on hold-out data: 0.2838050127029419
Epoch: 4
Loss on hold-out set: 2.3948677530828513
Accuracy on hold-out set: 0.14622641509433962
MAPK value on hold-out data: 0.28066033124923706
Returned to Spot: Validation loss: 2.3948677530828513
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#####-----] 46.12% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.019019571785415357, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3974973728071967
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.16981132328510284
Epoch: 2
Loss on hold-out set: 2.3969540393577433
Accuracy on hold-out set: 0.10849056603773585
MAPK value on hold-out data: 0.2154087871313095
Epoch: 3
Loss on hold-out set: 2.3962573600265213
Accuracy on hold-out set: 0.10849056603773585
MAPK value on hold-out data: 0.2083333283662796
Epoch: 4
Loss on hold-out set: 2.395260790608964
Accuracy on hold-out set: 0.1179245283018868
MAPK value on hold-out data: 0.2146226167678833
Returned to Spot: Validation loss: 2.395260790608964
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#####-----] 47.21% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.019297683186435734, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.397590086145221
Accuracy on hold-out set: 0.14622641509433962
MAPK value on hold-out data: 0.21147798001766205
Epoch: 2
Loss on hold-out set: 2.3970916248717398
Accuracy on hold-out set: 0.1320754716981132
MAPK value on hold-out data: 0.24528300762176514
Epoch: 3
Loss on hold-out set: 2.3964011736635893
Accuracy on hold-out set: 0.13679245283018868
MAPK value on hold-out data: 0.26022008061408997
Epoch: 4
Loss on hold-out set: 2.395342433227683
Accuracy on hold-out set: 0.16037735849056603
MAPK value on hold-out data: 0.26572322845458984
Returned to Spot: Validation loss: 2.395342433227683
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#####-----] 48.21% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.01908068498088259, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.397247501139371
Accuracy on hold-out set: 0.09433962264150944
MAPK value on hold-out data: 0.22955970466136932
Epoch: 2
Loss on hold-out set: 2.3966732992316193
Accuracy on hold-out set: 0.09433962264150944
MAPK value on hold-out data: 0.2413521707057953
Epoch: 3
Loss on hold-out set: 2.395920366611121
Accuracy on hold-out set: 0.09433962264150944
MAPK value on hold-out data: 0.23270437121391296
Epoch: 4
Loss on hold-out set: 2.3948610323779986
Accuracy on hold-out set: 0.09433962264150944
MAPK value on hold-out data: 0.21776728332042694
Returned to Spot: Validation loss: 2.3948610323779986
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#####-----] 49.12% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.019322323708321827, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.397585189567422
Accuracy on hold-out set: 0.1792452830188679
MAPK value on hold-out data: 0.27515721321105957
Epoch: 2
Loss on hold-out set: 2.3969848515852443
Accuracy on hold-out set: 0.19811320754716982
MAPK value on hold-out data: 0.29323896765708923
Epoch: 3
Loss on hold-out set: 2.3962404143135503
Accuracy on hold-out set: 0.2028301886792453
MAPK value on hold-out data: 0.30974841117858887
Epoch: 4
Loss on hold-out set: 2.395169258117676
Accuracy on hold-out set: 0.23113207547169812
MAPK value on hold-out data: 0.331760972738266
Returned to Spot: Validation loss: 2.395169258117676
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#####-----] 50.07% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.019352743323080308, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3974609375
Accuracy on hold-out set: 0.08490566037735849
MAPK value on hold-out data: 0.15172958374023438
Epoch: 2
Loss on hold-out set: 2.3969019291535862
Accuracy on hold-out set: 0.08962264150943396
MAPK value on hold-out data: 0.18317611515522003
Epoch: 3
Loss on hold-out set: 2.3961017199282377
Accuracy on hold-out set: 0.08962264150943396
MAPK value on hold-out data: 0.24371063709259033
Epoch: 4
Loss on hold-out set: 2.39492300321471
Accuracy on hold-out set: 0.09905660377358491
MAPK value on hold-out data: 0.2617923617362976
Returned to Spot: Validation loss: 2.39492300321471
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#####-----] 51.10% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.019337066800818842, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.397331624660852
Accuracy on hold-out set: 0.08018867924528301
MAPK value on hold-out data: 0.18238994479179382
Epoch: 2
Loss on hold-out set: 2.3966128826141357
Accuracy on hold-out set: 0.1320754716981132
MAPK value on hold-out data: 0.20361630618572235
Epoch: 3
Loss on hold-out set: 2.3957116109020307
Accuracy on hold-out set: 0.1792452830188679
MAPK value on hold-out data: 0.25157228112220764
Epoch: 4
Loss on hold-out set: 2.3943913680202558
Accuracy on hold-out set: 0.17452830188679244
MAPK value on hold-out data: 0.2641509175300598
Returned to Spot: Validation loss: 2.3943913680202558
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#####-----] 52.12% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.01931887940524818, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3972027189326734
Accuracy on hold-out set: 0.15566037735849056
MAPK value on hold-out data: 0.21305030584335327
Epoch: 2
Loss on hold-out set: 2.3963825702667236
Accuracy on hold-out set: 0.16037735849056603
MAPK value on hold-out data: 0.2437106817960739
Epoch: 3
Loss on hold-out set: 2.395384725534691
Accuracy on hold-out set: 0.14150943396226415
MAPK value on hold-out data: 0.2547169029712677
Epoch: 4
Loss on hold-out set: 2.393934168905582
Accuracy on hold-out set: 0.15566037735849056
MAPK value on hold-out data: 0.2814464867115021
Returned to Spot: Validation loss: 2.393934168905582
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#####-----] 53.10% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.01922810547445886, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3972563248760297
Accuracy on hold-out set: 0.09433962264150944
MAPK value on hold-out data: 0.22484275698661804
Epoch: 2
Loss on hold-out set: 2.3964952365407406
Accuracy on hold-out set: 0.1792452830188679
MAPK value on hold-out data: 0.31682389974594116
Epoch: 3
Loss on hold-out set: 2.3955198908751867
Accuracy on hold-out set: 0.21226415094339623
MAPK value on hold-out data: 0.34355348348617554
Epoch: 4
Loss on hold-out set: 2.394165423681151
Accuracy on hold-out set: 0.25
MAPK value on hold-out data: 0.3663522005081177
Returned to Spot: Validation loss: 2.394165423681151
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#####-----] 54.18% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.018671012730914024, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3971813017467283
Accuracy on hold-out set: 0.2169811320754717
MAPK value on hold-out data: 0.3270440101623535
Epoch: 2
Loss on hold-out set: 2.3964601170341924
Accuracy on hold-out set: 0.2169811320754717
MAPK value on hold-out data: 0.3199685215950012
Epoch: 3
Loss on hold-out set: 2.3955165979997166
Accuracy on hold-out set: 0.23113207547169812
MAPK value on hold-out data: 0.33411943912506104
Epoch: 4
Loss on hold-out set: 2.394212356153524
Accuracy on hold-out set: 0.2358490566037736
MAPK value on hold-out data: 0.3396225571632385
Returned to Spot: Validation loss: 2.394212356153524
----------------------------------------------
spotPython tuning: 2.3929684499524675 [######----] 55.28% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.018247371903514324, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.397546367825202
Accuracy on hold-out set: 0.09905660377358491
MAPK value on hold-out data: 0.18867923319339752
Epoch: 2
Loss on hold-out set: 2.3969439403066097
Accuracy on hold-out set: 0.09905660377358491
MAPK value on hold-out data: 0.1996854990720749
Epoch: 3
Loss on hold-out set: 2.3960605544864007
Accuracy on hold-out set: 0.09905660377358491
MAPK value on hold-out data: 0.21069177985191345
Epoch: 4
Loss on hold-out set: 2.3947962994845406
Accuracy on hold-out set: 0.09905660377358491
MAPK value on hold-out data: 0.21226412057876587
Returned to Spot: Validation loss: 2.3947962994845406
----------------------------------------------
spotPython tuning: 2.3929684499524675 [######----] 56.36% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.01849021323967576, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3973293416904955
Accuracy on hold-out set: 0.0660377358490566
MAPK value on hold-out data: 0.16509436070919037
Epoch: 2
Loss on hold-out set: 2.396642061899293
Accuracy on hold-out set: 0.1509433962264151
MAPK value on hold-out data: 0.2484276294708252
Epoch: 3
Loss on hold-out set: 2.3957265165616883
Accuracy on hold-out set: 0.2358490566037736
MAPK value on hold-out data: 0.3349056541919708
Epoch: 4
Loss on hold-out set: 2.394485869497623
Accuracy on hold-out set: 0.25943396226415094
MAPK value on hold-out data: 0.3411949872970581
Returned to Spot: Validation loss: 2.394485869497623
----------------------------------------------
spotPython tuning: 2.3929684499524675 [######----] 57.58% 

config: {'_L0': 6112, 'l1': 256, 'dropout_prob': 0.23276754880203154, 'lr_mult': 0.001, 'batch_size': 4, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'Adamax', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3972731626258708
Accuracy on hold-out set: 0.1320754716981132
MAPK value on hold-out data: 0.17531447112560272
Epoch: 2
Loss on hold-out set: 2.3972305036940664
Accuracy on hold-out set: 0.12735849056603774
MAPK value on hold-out data: 0.1721697896718979
Epoch: 3
Loss on hold-out set: 2.397210287597944
Accuracy on hold-out set: 0.1320754716981132
MAPK value on hold-out data: 0.17845910787582397
Epoch: 4
Loss on hold-out set: 2.3972248986082256
Accuracy on hold-out set: 0.1320754716981132
MAPK value on hold-out data: 0.17688679695129395
Returned to Spot: Validation loss: 2.3972248986082256
----------------------------------------------
spotPython tuning: 2.3929684499524675 [######----] 58.15% 

config: {'_L0': 6112, 'l1': 256, 'dropout_prob': 0.2431173831408648, 'lr_mult': 0.001, 'batch_size': 16, 'epochs': 4, 'k_folds': 1, 'patience': 8, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3983785084315707
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.1778273582458496
Epoch: 2
Loss on hold-out set: 2.398346083504813
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.1800595372915268
Epoch: 3
Loss on hold-out set: 2.3983379432133267
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.1770833432674408
Epoch: 4
Loss on hold-out set: 2.3984008516584123
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.1800595074892044
Returned to Spot: Validation loss: 2.3984008516584123
----------------------------------------------
spotPython tuning: 2.3929684499524675 [######----] 58.67% 

config: {'_L0': 6112, 'l1': 64, 'dropout_prob': 0.01513589825644096, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 64, 'optimizer': 'Adam', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.400147039935274
Accuracy on hold-out set: 0.0660377358490566
MAPK value on hold-out data: 0.14937108755111694
Epoch: 2
Loss on hold-out set: 2.400128555747698
Accuracy on hold-out set: 0.0660377358490566
MAPK value on hold-out data: 0.14937108755111694
Epoch: 3
Loss on hold-out set: 2.4001237761299565
Accuracy on hold-out set: 0.0660377358490566
MAPK value on hold-out data: 0.14937108755111694
Epoch: 4
Loss on hold-out set: 2.4001118709456244
Accuracy on hold-out set: 0.0660377358490566
MAPK value on hold-out data: 0.14937108755111694
Returned to Spot: Validation loss: 2.4001118709456244
----------------------------------------------
spotPython tuning: 2.3929684499524675 [######----] 59.60% 

config: {'_L0': 6112, 'l1': 4096, 'dropout_prob': 0.31016441943384926, 'lr_mult': 0.001, 'batch_size': 16, 'epochs': 4, 'k_folds': 1, 'patience': 32, 'optimizer': 'NAdam', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.397723231996809
Accuracy on hold-out set: 0.11320754716981132
MAPK value on hold-out data: 0.197172611951828
Epoch: 2
Loss on hold-out set: 2.3976612431662425
Accuracy on hold-out set: 0.09433962264150944
MAPK value on hold-out data: 0.1755952537059784
Epoch: 3
Loss on hold-out set: 2.397666811943054
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.1808035522699356
Epoch: 4
Loss on hold-out set: 2.397546018872942
Accuracy on hold-out set: 0.09433962264150944
MAPK value on hold-out data: 0.1867559403181076
Returned to Spot: Validation loss: 2.397546018872942
----------------------------------------------
spotPython tuning: 2.3929684499524675 [######----] 60.39% 

config: {'_L0': 6112, 'l1': 512, 'dropout_prob': 0.10123541888783366, 'lr_mult': 0.001, 'batch_size': 8, 'epochs': 4, 'k_folds': 1, 'patience': 32, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.398464723869606
Accuracy on hold-out set: 0.07075471698113207
MAPK value on hold-out data: 0.14506173133850098
Epoch: 2
Loss on hold-out set: 2.398421084439313
Accuracy on hold-out set: 0.07075471698113207
MAPK value on hold-out data: 0.14506173133850098
Epoch: 3
Loss on hold-out set: 2.3984013398488364
Accuracy on hold-out set: 0.07075471698113207
MAPK value on hold-out data: 0.14429013431072235
Epoch: 4
Loss on hold-out set: 2.3983712991078696
Accuracy on hold-out set: 0.07075471698113207
MAPK value on hold-out data: 0.14429013431072235
Returned to Spot: Validation loss: 2.3983712991078696
----------------------------------------------
spotPython tuning: 2.3929684499524675 [######----] 61.67% 

config: {'_L0': 6112, 'l1': 512, 'dropout_prob': 0.42913010110548067, 'lr_mult': 0.001, 'batch_size': 4, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'Adamax', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.398660812737807
Accuracy on hold-out set: 0.09905660377358491
MAPK value on hold-out data: 0.1399371325969696
Epoch: 2
Loss on hold-out set: 2.3985173567286076
Accuracy on hold-out set: 0.09905660377358491
MAPK value on hold-out data: 0.1462264358997345
Epoch: 3
Loss on hold-out set: 2.398478539484852
Accuracy on hold-out set: 0.09905660377358491
MAPK value on hold-out data: 0.14386793971061707
Epoch: 4
Loss on hold-out set: 2.398469389609571
Accuracy on hold-out set: 0.09905660377358491
MAPK value on hold-out data: 0.14308176934719086
Returned to Spot: Validation loss: 2.398469389609571
----------------------------------------------
spotPython tuning: 2.3929684499524675 [######----] 62.78% 

config: {'_L0': 6112, 'l1': 2048, 'dropout_prob': 0.3275022803848954, 'lr_mult': 0.001, 'batch_size': 16, 'epochs': 4, 'k_folds': 1, 'patience': 8, 'optimizer': 'NAdam', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.397666232926505
Accuracy on hold-out set: 0.10849056603773585
MAPK value on hold-out data: 0.194196417927742
Epoch: 2
Loss on hold-out set: 2.3976151772907803
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.193452388048172
Epoch: 3
Loss on hold-out set: 2.3975475004741122
Accuracy on hold-out set: 0.09905660377358491
MAPK value on hold-out data: 0.1986607015132904
Epoch: 4
Loss on hold-out set: 2.397498760904585
Accuracy on hold-out set: 0.09905660377358491
MAPK value on hold-out data: 0.200892835855484
Returned to Spot: Validation loss: 2.397498760904585
----------------------------------------------
spotPython tuning: 2.3929684499524675 [######----] 63.73% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.015932020350763642, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3975161044102795
Accuracy on hold-out set: 0.09433962264150944
MAPK value on hold-out data: 0.19103772938251495
Epoch: 2
Loss on hold-out set: 2.39694562947975
Accuracy on hold-out set: 0.09905660377358491
MAPK value on hold-out data: 0.18238994479179382
Epoch: 3
Loss on hold-out set: 2.3962241928532437
Accuracy on hold-out set: 0.11320754716981132
MAPK value on hold-out data: 0.18238994479179382
Epoch: 4
Loss on hold-out set: 2.3951578185243427
Accuracy on hold-out set: 0.12735849056603774
MAPK value on hold-out data: 0.19103772938251495
Returned to Spot: Validation loss: 2.3951578185243427
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#######---] 65.51% 

config: {'_L0': 6112, 'l1': 128, 'dropout_prob': 0.46996641442364834, 'lr_mult': 0.001, 'batch_size': 4, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.399120735672285
Accuracy on hold-out set: 0.08490566037735849
MAPK value on hold-out data: 0.1312893182039261
Epoch: 2
Loss on hold-out set: 2.399069516163952
Accuracy on hold-out set: 0.08018867924528301
MAPK value on hold-out data: 0.12264152616262436
Epoch: 3
Loss on hold-out set: 2.3990387241795377
Accuracy on hold-out set: 0.08490566037735849
MAPK value on hold-out data: 0.12814466655254364
Epoch: 4
Loss on hold-out set: 2.3989528305125685
Accuracy on hold-out set: 0.08490566037735849
MAPK value on hold-out data: 0.12657234072685242
Returned to Spot: Validation loss: 2.3989528305125685
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#######---] 66.72% 

config: {'_L0': 6112, 'l1': 256, 'dropout_prob': 0.009699514739187665, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'NAdam', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3979219265703886
Accuracy on hold-out set: 0.1179245283018868
MAPK value on hold-out data: 0.17138366401195526
Epoch: 2
Loss on hold-out set: 2.3978162104228757
Accuracy on hold-out set: 0.1179245283018868
MAPK value on hold-out data: 0.16981133818626404
Epoch: 3
Loss on hold-out set: 2.3976900015237197
Accuracy on hold-out set: 0.1179245283018868
MAPK value on hold-out data: 0.16981133818626404
Epoch: 4
Loss on hold-out set: 2.397555020620238
Accuracy on hold-out set: 0.1179245283018868
MAPK value on hold-out data: 0.17138366401195526
Returned to Spot: Validation loss: 2.397555020620238
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#######---] 68.28% 

config: {'_L0': 6112, 'l1': 256, 'dropout_prob': 0.1397217765002243, 'lr_mult': 0.001, 'batch_size': 4, 'epochs': 4, 'k_folds': 1, 'patience': 4, 'optimizer': 'Adam', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.397415332074435
Accuracy on hold-out set: 0.13679245283018868
MAPK value on hold-out data: 0.1855345517396927
Epoch: 2
Loss on hold-out set: 2.39739487306127
Accuracy on hold-out set: 0.13679245283018868
MAPK value on hold-out data: 0.18867923319339752
Epoch: 3
Loss on hold-out set: 2.397396240594252
Accuracy on hold-out set: 0.1320754716981132
MAPK value on hold-out data: 0.18396224081516266
Epoch: 4
Loss on hold-out set: 2.397361660903355
Accuracy on hold-out set: 0.14150943396226415
MAPK value on hold-out data: 0.1926100254058838
Returned to Spot: Validation loss: 2.397361660903355
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#######---] 69.83% 

config: {'_L0': 6112, 'l1': 2048, 'dropout_prob': 0.11001746050022188, 'lr_mult': 0.001, 'batch_size': 4, 'epochs': 4, 'k_folds': 1, 'patience': 64, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3977913361675336
Accuracy on hold-out set: 0.10849056603773585
MAPK value on hold-out data: 0.17845909297466278
Epoch: 2
Loss on hold-out set: 2.397703805059757
Accuracy on hold-out set: 0.10849056603773585
MAPK value on hold-out data: 0.18160375952720642
Epoch: 3
Loss on hold-out set: 2.3976109162816464
Accuracy on hold-out set: 0.10849056603773585
MAPK value on hold-out data: 0.18946537375450134
Epoch: 4
Loss on hold-out set: 2.3975684912699573
Accuracy on hold-out set: 0.10849056603773585
MAPK value on hold-out data: 0.1863207370042801
Returned to Spot: Validation loss: 2.3975684912699573
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#######---] 71.36% 

config: {'_L0': 6112, 'l1': 512, 'dropout_prob': 0.4172288580201352, 'lr_mult': 0.001, 'batch_size': 4, 'epochs': 4, 'k_folds': 1, 'patience': 8, 'optimizer': 'Adamax', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.398982866755072
Accuracy on hold-out set: 0.07075471698113207
MAPK value on hold-out data: 0.13679245114326477
Epoch: 2
Loss on hold-out set: 2.3989910584575727
Accuracy on hold-out set: 0.07075471698113207
MAPK value on hold-out data: 0.13364779949188232
Epoch: 3
Loss on hold-out set: 2.39894186325793
Accuracy on hold-out set: 0.07075471698113207
MAPK value on hold-out data: 0.1320754736661911
Epoch: 4
Loss on hold-out set: 2.398926797902809
Accuracy on hold-out set: 0.07075471698113207
MAPK value on hold-out data: 0.1320754736661911
Returned to Spot: Validation loss: 2.398926797902809
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#######---] 72.53% 

config: {'_L0': 6112, 'l1': 64, 'dropout_prob': 0.27704058285336897, 'lr_mult': 0.001, 'batch_size': 4, 'epochs': 4, 'k_folds': 1, 'patience': 64, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3998542911601515
Accuracy on hold-out set: 0.09905660377358491
MAPK value on hold-out data: 0.15566037595272064
Epoch: 2
Loss on hold-out set: 2.3997569354075305
Accuracy on hold-out set: 0.09905660377358491
MAPK value on hold-out data: 0.15644653141498566
Epoch: 3
Loss on hold-out set: 2.39974130774444
Accuracy on hold-out set: 0.09433962264150944
MAPK value on hold-out data: 0.15015724301338196
Epoch: 4
Loss on hold-out set: 2.39979805586473
Accuracy on hold-out set: 0.09433962264150944
MAPK value on hold-out data: 0.151729553937912
Returned to Spot: Validation loss: 2.39979805586473
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#######---] 73.45% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.011840236923638793, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3971494323802442
Accuracy on hold-out set: 0.12735849056603774
MAPK value on hold-out data: 0.22484271228313446
Epoch: 2
Loss on hold-out set: 2.3964147477779747
Accuracy on hold-out set: 0.1320754716981132
MAPK value on hold-out data: 0.25314462184906006
Epoch: 3
Loss on hold-out set: 2.395440443506781
Accuracy on hold-out set: 0.1179245283018868
MAPK value on hold-out data: 0.2547169327735901
Epoch: 4
Loss on hold-out set: 2.394071844388854
Accuracy on hold-out set: 0.1509433962264151
MAPK value on hold-out data: 0.2900942265987396
Returned to Spot: Validation loss: 2.394071844388854
----------------------------------------------
spotPython tuning: 2.3929684499524675 [########--] 75.06% 

config: {'_L0': 6112, 'l1': 128, 'dropout_prob': 0.36895509123433334, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 8, 'optimizer': 'Adam', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3979437531165355
Accuracy on hold-out set: 0.06132075471698113
MAPK value on hold-out data: 0.1666666716337204
Epoch: 2
Loss on hold-out set: 2.3979133502492367
Accuracy on hold-out set: 0.06132075471698113
MAPK value on hold-out data: 0.1666666716337204
Epoch: 3
Loss on hold-out set: 2.3978111698942364
Accuracy on hold-out set: 0.06132075471698113
MAPK value on hold-out data: 0.1666666716337204
Epoch: 4
Loss on hold-out set: 2.3979258469815523
Accuracy on hold-out set: 0.06132075471698113
MAPK value on hold-out data: 0.16509434580802917
Returned to Spot: Validation loss: 2.3979258469815523
----------------------------------------------
spotPython tuning: 2.3929684499524675 [########--] 76.54% 

config: {'_L0': 6112, 'l1': 2048, 'dropout_prob': 0.7935318995459109, 'lr_mult': 0.001, 'batch_size': 16, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'Adamax', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.39801972252982
Accuracy on hold-out set: 0.11320754716981132
MAPK value on hold-out data: 0.1927083283662796
Epoch: 2
Loss on hold-out set: 2.398078509739467
Accuracy on hold-out set: 0.08018867924528301
MAPK value on hold-out data: 0.1577380895614624
Epoch: 3
Loss on hold-out set: 2.397971885544913
Accuracy on hold-out set: 0.12264150943396226
MAPK value on hold-out data: 0.1986607164144516
Epoch: 4
Loss on hold-out set: 2.3980720724378313
Accuracy on hold-out set: 0.1179245283018868
MAPK value on hold-out data: 0.1897321492433548
Returned to Spot: Validation loss: 2.3980720724378313
----------------------------------------------
spotPython tuning: 2.3929684499524675 [########--] 77.82% 

config: {'_L0': 6112, 'l1': 1024, 'dropout_prob': 0.4710472795678196, 'lr_mult': 0.001, 'batch_size': 8, 'epochs': 4, 'k_folds': 1, 'patience': 64, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3977493621684887
Accuracy on hold-out set: 0.04716981132075472
MAPK value on hold-out data: 0.1250000149011612
Epoch: 2
Loss on hold-out set: 2.397824110808196
Accuracy on hold-out set: 0.04716981132075472
MAPK value on hold-out data: 0.1280864179134369
Epoch: 3
Loss on hold-out set: 2.3977877298990884
Accuracy on hold-out set: 0.06132075471698113
MAPK value on hold-out data: 0.1350308656692505
Epoch: 4
Loss on hold-out set: 2.397755234329789
Accuracy on hold-out set: 0.05188679245283019
MAPK value on hold-out data: 0.13117283582687378
Returned to Spot: Validation loss: 2.397755234329789
----------------------------------------------
spotPython tuning: 2.3929684499524675 [########--] 78.89% 

config: {'_L0': 6112, 'l1': 2048, 'dropout_prob': 0.8195974031663599, 'lr_mult': 0.001, 'batch_size': 4, 'epochs': 4, 'k_folds': 1, 'patience': 8, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3980359581281556
Accuracy on hold-out set: 0.09905660377358491
MAPK value on hold-out data: 0.16116352379322052
Epoch: 2
Loss on hold-out set: 2.397875232516595
Accuracy on hold-out set: 0.09433962264150944
MAPK value on hold-out data: 0.16273587942123413
Epoch: 3
Loss on hold-out set: 2.397877427766908
Accuracy on hold-out set: 0.08490566037735849
MAPK value on hold-out data: 0.1540880799293518
Epoch: 4
Loss on hold-out set: 2.397973753371329
Accuracy on hold-out set: 0.09433962264150944
MAPK value on hold-out data: 0.16588051617145538
Returned to Spot: Validation loss: 2.397973753371329
----------------------------------------------
spotPython tuning: 2.3929684499524675 [########--] 80.18% 

config: {'_L0': 6112, 'l1': 4096, 'dropout_prob': 0.485316841383955, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 32, 'optimizer': 'Adamax', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3977512053723604
Accuracy on hold-out set: 0.08018867924528301
MAPK value on hold-out data: 0.16273584961891174
Epoch: 2
Loss on hold-out set: 2.3977309510392963
Accuracy on hold-out set: 0.07547169811320754
MAPK value on hold-out data: 0.16273584961891174
Epoch: 3
Loss on hold-out set: 2.397669198378077
Accuracy on hold-out set: 0.0660377358490566
MAPK value on hold-out data: 0.1603773832321167
Epoch: 4
Loss on hold-out set: 2.3975497371745558
Accuracy on hold-out set: 0.07075471698113207
MAPK value on hold-out data: 0.16902516782283783
Returned to Spot: Validation loss: 2.3975497371745558
----------------------------------------------
spotPython tuning: 2.3929684499524675 [########--] 81.58% 

config: {'_L0': 6112, 'l1': 512, 'dropout_prob': 0.5521541276635835, 'lr_mult': 0.001, 'batch_size': 4, 'epochs': 4, 'k_folds': 1, 'patience': 64, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3971631481962383
Accuracy on hold-out set: 0.07075471698113207
MAPK value on hold-out data: 0.16116352379322052
Epoch: 2
Loss on hold-out set: 2.3971064720513686
Accuracy on hold-out set: 0.07075471698113207
MAPK value on hold-out data: 0.17767292261123657
Epoch: 3
Loss on hold-out set: 2.3971460540339633
Accuracy on hold-out set: 0.0660377358490566
MAPK value on hold-out data: 0.16509436070919037
Epoch: 4
Loss on hold-out set: 2.397186135346035
Accuracy on hold-out set: 0.07075471698113207
MAPK value on hold-out data: 0.15566037595272064
Returned to Spot: Validation loss: 2.397186135346035
----------------------------------------------
spotPython tuning: 2.3929684499524675 [########--] 82.78% 

config: {'_L0': 6112, 'l1': 64, 'dropout_prob': 0.4286259094831192, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'Adam', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.397110941275111
Accuracy on hold-out set: 0.11320754716981132
MAPK value on hold-out data: 0.17610064148902893
Epoch: 2
Loss on hold-out set: 2.3969799347643583
Accuracy on hold-out set: 0.11320754716981132
MAPK value on hold-out data: 0.17767298221588135
Epoch: 3
Loss on hold-out set: 2.3970077172765194
Accuracy on hold-out set: 0.1179245283018868
MAPK value on hold-out data: 0.1816037893295288
Epoch: 4
Loss on hold-out set: 2.396970906347599
Accuracy on hold-out set: 0.11320754716981132
MAPK value on hold-out data: 0.18238995969295502
Returned to Spot: Validation loss: 2.396970906347599
----------------------------------------------
spotPython tuning: 2.3929684499524675 [########--] 84.10% 

config: {'_L0': 6112, 'l1': 2048, 'dropout_prob': 0.4976005796813294, 'lr_mult': 0.001, 'batch_size': 8, 'epochs': 4, 'k_folds': 1, 'patience': 4, 'optimizer': 'NAdam', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3978147506713867
Accuracy on hold-out set: 0.09905660377358491
MAPK value on hold-out data: 0.15817902982234955
Epoch: 2
Loss on hold-out set: 2.3976941108703613
Accuracy on hold-out set: 0.09905660377358491
MAPK value on hold-out data: 0.17592591047286987
Epoch: 3
Loss on hold-out set: 2.3976512220170765
Accuracy on hold-out set: 0.08962264150943396
MAPK value on hold-out data: 0.17438271641731262
Epoch: 4
Loss on hold-out set: 2.3975630159731263
Accuracy on hold-out set: 0.09905660377358491
MAPK value on hold-out data: 0.1736111044883728
Returned to Spot: Validation loss: 2.3975630159731263
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#########-] 85.51% 

config: {'_L0': 6112, 'l1': 1024, 'dropout_prob': 0.14496135146071007, 'lr_mult': 0.001, 'batch_size': 8, 'epochs': 4, 'k_folds': 1, 'patience': 32, 'optimizer': 'NAdam', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3977424921812833
Accuracy on hold-out set: 0.08962264150943396
MAPK value on hold-out data: 0.16820989549160004
Epoch: 2
Loss on hold-out set: 2.39760242568122
Accuracy on hold-out set: 0.11320754716981132
MAPK value on hold-out data: 0.17901232838630676
Epoch: 3
Loss on hold-out set: 2.397585391998291
Accuracy on hold-out set: 0.10849056603773585
MAPK value on hold-out data: 0.1766975224018097
Epoch: 4
Loss on hold-out set: 2.3975139635580556
Accuracy on hold-out set: 0.09905660377358491
MAPK value on hold-out data: 0.17592592537403107
Returned to Spot: Validation loss: 2.3975139635580556
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#########-] 86.79% 

config: {'_L0': 6112, 'l1': 256, 'dropout_prob': 0.13836940708432457, 'lr_mult': 0.001, 'batch_size': 4, 'epochs': 4, 'k_folds': 1, 'patience': 32, 'optimizer': 'Adamax', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3983789704880625
Accuracy on hold-out set: 0.08490566037735849
MAPK value on hold-out data: 0.1603773832321167
Epoch: 2
Loss on hold-out set: 2.398338848689817
Accuracy on hold-out set: 0.08490566037735849
MAPK value on hold-out data: 0.15723270177841187
Epoch: 3
Loss on hold-out set: 2.3983498924183397
Accuracy on hold-out set: 0.08490566037735849
MAPK value on hold-out data: 0.15880504250526428
Epoch: 4
Loss on hold-out set: 2.398287737144614
Accuracy on hold-out set: 0.08018867924528301
MAPK value on hold-out data: 0.16116352379322052
Returned to Spot: Validation loss: 2.398287737144614
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#########-] 88.07% 

config: {'_L0': 6112, 'l1': 1024, 'dropout_prob': 0.5801902947427491, 'lr_mult': 0.001, 'batch_size': 4, 'epochs': 4, 'k_folds': 1, 'patience': 8, 'optimizer': 'Adamax', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.398075229716751
Accuracy on hold-out set: 0.07547169811320754
MAPK value on hold-out data: 0.14937110245227814
Epoch: 2
Loss on hold-out set: 2.3982018569730363
Accuracy on hold-out set: 0.08018867924528301
MAPK value on hold-out data: 0.14308178424835205
Epoch: 3
Loss on hold-out set: 2.398067883725436
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.16116352379322052
Epoch: 4
Loss on hold-out set: 2.398173395192848
Accuracy on hold-out set: 0.08018867924528301
MAPK value on hold-out data: 0.1470125913619995
Returned to Spot: Validation loss: 2.398173395192848
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#########-] 89.23% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.01611888506451225, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3972911137454913
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.19261004030704498
Epoch: 2
Loss on hold-out set: 2.3966208763842314
Accuracy on hold-out set: 0.12264150943396226
MAPK value on hold-out data: 0.21698109805583954
Epoch: 3
Loss on hold-out set: 2.3957599491443275
Accuracy on hold-out set: 0.1320754716981132
MAPK value on hold-out data: 0.23977987468242645
Epoch: 4
Loss on hold-out set: 2.394550366221734
Accuracy on hold-out set: 0.14622641509433962
MAPK value on hold-out data: 0.27279871702194214
Returned to Spot: Validation loss: 2.394550366221734
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#########-] 90.97% 

config: {'_L0': 6112, 'l1': 64, 'dropout_prob': 0.6944080618643941, 'lr_mult': 0.001, 'batch_size': 8, 'epochs': 4, 'k_folds': 1, 'patience': 8, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3952093477602356
Accuracy on hold-out set: 0.09433962264150944
MAPK value on hold-out data: 0.18441356718540192
Epoch: 2
Loss on hold-out set: 2.3952625415943287
Accuracy on hold-out set: 0.08962264150943396
MAPK value on hold-out data: 0.18364198505878448
Epoch: 3
Loss on hold-out set: 2.395083259653162
Accuracy on hold-out set: 0.09905660377358491
MAPK value on hold-out data: 0.1875
Epoch: 4
Loss on hold-out set: 2.3950424635851824
Accuracy on hold-out set: 0.08962264150943396
MAPK value on hold-out data: 0.18981482088565826
Returned to Spot: Validation loss: 2.3950424635851824
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#########-] 92.32% 

config: {'_L0': 6112, 'l1': 128, 'dropout_prob': 0.18151742556849865, 'lr_mult': 0.001, 'batch_size': 16, 'epochs': 4, 'k_folds': 1, 'patience': 64, 'optimizer': 'Adam', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3986207246780396
Accuracy on hold-out set: 0.08962264150943396
MAPK value on hold-out data: 0.169642835855484
Epoch: 2
Loss on hold-out set: 2.3985854046685353
Accuracy on hold-out set: 0.09433962264150944
MAPK value on hold-out data: 0.1718749850988388
Epoch: 3
Loss on hold-out set: 2.398624931062971
Accuracy on hold-out set: 0.10849056603773585
MAPK value on hold-out data: 0.1800595223903656
Epoch: 4
Loss on hold-out set: 2.3986248629433766
Accuracy on hold-out set: 0.09433962264150944
MAPK value on hold-out data: 0.1733630895614624
Returned to Spot: Validation loss: 2.3986248629433766
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#########-] 93.68% 

config: {'_L0': 6112, 'l1': 128, 'dropout_prob': 0.845806549101736, 'lr_mult': 0.001, 'batch_size': 8, 'epochs': 4, 'k_folds': 1, 'patience': 8, 'optimizer': 'NAdam', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3980945922710277
Accuracy on hold-out set: 0.08962264150943396
MAPK value on hold-out data: 0.19290123879909515
Epoch: 2
Loss on hold-out set: 2.3980139979609736
Accuracy on hold-out set: 0.12264150943396226
MAPK value on hold-out data: 0.1983024775981903
Epoch: 3
Loss on hold-out set: 2.398009423856382
Accuracy on hold-out set: 0.08490566037735849
MAPK value on hold-out data: 0.19598767161369324
Epoch: 4
Loss on hold-out set: 2.3981512475896767
Accuracy on hold-out set: 0.09433962264150944
MAPK value on hold-out data: 0.1952160894870758
Returned to Spot: Validation loss: 2.3981512475896767
----------------------------------------------
spotPython tuning: 2.3929684499524675 [#########-] 94.82% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.010755424290215925, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3972837969941914
Accuracy on hold-out set: 0.17452830188679244
MAPK value on hold-out data: 0.24764150381088257
Epoch: 2
Loss on hold-out set: 2.396708607673645
Accuracy on hold-out set: 0.1650943396226415
MAPK value on hold-out data: 0.2570754587650299
Epoch: 3
Loss on hold-out set: 2.395920512811193
Accuracy on hold-out set: 0.1509433962264151
MAPK value on hold-out data: 0.2641509473323822
Epoch: 4
Loss on hold-out set: 2.3947488204488216
Accuracy on hold-out set: 0.14150943396226415
MAPK value on hold-out data: 0.26650938391685486
Returned to Spot: Validation loss: 2.3947488204488216
----------------------------------------------
spotPython tuning: 2.3929684499524675 [##########] 96.55% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.011912050401846183, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.397292382312271
Accuracy on hold-out set: 0.12264150943396226
MAPK value on hold-out data: 0.22248423099517822
Epoch: 2
Loss on hold-out set: 2.3966105524099097
Accuracy on hold-out set: 0.18396226415094338
MAPK value on hold-out data: 0.2845911383628845
Epoch: 3
Loss on hold-out set: 2.395679572843156
Accuracy on hold-out set: 0.18867924528301888
MAPK value on hold-out data: 0.2963835299015045
Epoch: 4
Loss on hold-out set: 2.394390160182737
Accuracy on hold-out set: 0.19811320754716982
MAPK value on hold-out data: 0.3050313889980316
Returned to Spot: Validation loss: 2.394390160182737
----------------------------------------------
spotPython tuning: 2.3929684499524675 [##########] 98.35% 

config: {'_L0': 6112, 'l1': 8192, 'dropout_prob': 0.010045707520406318, 'lr_mult': 0.001, 'batch_size': 2, 'epochs': 4, 'k_folds': 1, 'patience': 16, 'optimizer': 'AdamW', 'sgd_momentum': 0.9}
Epoch: 1
Loss on hold-out set: 2.3973825922552146
Accuracy on hold-out set: 0.1509433962264151
MAPK value on hold-out data: 0.23663519322872162
Epoch: 2
Loss on hold-out set: 2.396780205222796
Accuracy on hold-out set: 0.17452830188679244
MAPK value on hold-out data: 0.2759433388710022
Epoch: 3
Loss on hold-out set: 2.3959679288684197
Accuracy on hold-out set: 0.16981132075471697
MAPK value on hold-out data: 0.2759433686733246
Epoch: 4
Loss on hold-out set: 2.394827442349128
Accuracy on hold-out set: 0.1509433962264151
MAPK value on hold-out data: 0.2633647620677948
Returned to Spot: Validation loss: 2.394827442349128
----------------------------------------------
spotPython tuning: 2.3929684499524675 [##########] 100.00% Done...
<spotPython.spot.spot.Spot at 0x151de03a0>

21.6 Tensorboard

The textual output shown in the console (or code cell) can be visualized with Tensorboard as described in Section 13.13.

21.7 Results

After the hyperparameter tuning run is finished, the results can be analyzed as described in Section 13.14.

spot_tuner.plot_progress(log_y=False, 
    filename="./figures/" + experiment_name+"_progress.png")

Progress plot. Black dots denote results from the initial design. Red dots illustrate the improvement found by the surrogate model based optimization.
from spotPython.utils.eda import gen_design_table
print(gen_design_table(fun_control=fun_control, spot=spot_tuner))
| name         | type   | default   |   lower |   upper |               tuned | transform             |   importance | stars   |
|--------------|--------|-----------|---------|---------|---------------------|-----------------------|--------------|---------|
| _L0          | int    | 64        |  6112.0 |  6112.0 |              6112.0 | None                  |         0.00 |         |
| l1           | int    | 8         |     6.0 |    13.0 |                13.0 | transform_power_2_int |       100.00 | ***     |
| dropout_prob | float  | 0.01      |     0.0 |     0.9 | 0.40178776340101174 | None                  |        33.81 | *       |
| lr_mult      | float  | 1.0       |   0.001 |   0.001 |               0.001 | None                  |         0.00 |         |
| batch_size   | int    | 4         |     1.0 |     4.0 |                 1.0 | transform_power_2_int |        10.72 | *       |
| epochs       | int    | 4         |     2.0 |     2.0 |                 2.0 | transform_power_2_int |         0.00 |         |
| k_folds      | int    | 1         |     1.0 |     1.0 |                 1.0 | None                  |         0.00 |         |
| patience     | int    | 2         |     2.0 |     6.0 |                 4.0 | transform_power_2_int |         2.75 | *       |
| optimizer    | factor | SGD       |     0.0 |     3.0 |                 3.0 | None                  |        11.88 | *       |
| sgd_momentum | float  | 0.0       |     0.9 |     0.9 |                 0.9 | None                  |         0.00 |         |
spot_tuner.plot_importance(threshold=0.025,
    filename="./figures/" + experiment_name+"_importance.png")

Variable importance plot, threshold 0.025.

21.8 Get the Tuned Architecture

from spotPython.hyperparameters.values import get_one_core_model_from_X
X = spot_tuner.to_all_dim(spot_tuner.min_X.reshape(1,-1))
model_spot = get_one_core_model_from_X(X, fun_control)
model_spot
Net_vbdp(
  (fc1): Linear(in_features=6112, out_features=8192, bias=True)
  (fc2): Linear(in_features=8192, out_features=4096, bias=True)
  (fc3): Linear(in_features=4096, out_features=2048, bias=True)
  (fc4): Linear(in_features=2048, out_features=1024, bias=True)
  (fc5): Linear(in_features=1024, out_features=11, bias=True)
  (relu): ReLU()
  (softmax): Softmax(dim=1)
  (dropout1): Dropout(p=0.40178776340101174, inplace=False)
  (dropout2): Dropout(p=0.20089388170050587, inplace=False)
)

21.9 Evaluation of the Tuned Architecture

from spotPython.torch.traintest import (
    train_tuned,
    test_tuned,
    )
train_tuned(net=model_spot, train_dataset=train,
        loss_function=fun_control["loss_function"],
        metric=fun_control["metric_torch"],
        shuffle=True,
        device = fun_control["device"],
        path=None,
        task=fun_control["task"],)
Epoch: 1
Loss on hold-out set: 2.3973280101452232
Accuracy on hold-out set: 0.11320754716981132
MAPK value on hold-out data: 0.21069182455539703
Epoch: 2
Loss on hold-out set: 2.3964705377254845
Accuracy on hold-out set: 0.11320754716981132
MAPK value on hold-out data: 0.1933962106704712
Epoch: 3
Loss on hold-out set: 2.3948900789584755
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.18946541845798492
Epoch: 4
Loss on hold-out set: 2.3921453795343077
Accuracy on hold-out set: 0.10377358490566038
MAPK value on hold-out data: 0.19182388484477997
Returned to Spot: Validation loss: 2.3921453795343077
----------------------------------------------

If path is set to a filename, e.g., path = "model_spot_trained.pt", the weights of the trained model will be loaded from this file.

test_tuned(net=model_spot, test_dataset=test,
            shuffle=False,
            loss_function=fun_control["loss_function"],
            metric=fun_control["metric_torch"],
            device = fun_control["device"],
            task=fun_control["task"],)
Loss on hold-out set: 2.390492484810647
Accuracy on hold-out set: 0.11864406779661017
MAPK value on hold-out data: 0.21254681050777435
Final evaluation: Validation loss: 2.390492484810647
Final evaluation: Validation metric: 0.21254681050777435
----------------------------------------------
(2.390492484810647, nan, tensor(0.2125))

21.10 Cross-validated Evaluations

  • This is the evaluation that will be used in the comparison (evaluatecv has to be updated before, to get metric vlaues!):
from spotPython.torch.traintest import evaluate_cv
# modify k-kolds:
setattr(model_spot, "k_folds",  10)
df_eval, df_preds, df_metrics = evaluate_cv(net=model_spot,
    dataset=fun_control["data"],
    loss_function=fun_control["loss_function"],
    metric=fun_control["metric_torch"],
    task=fun_control["task"],
    writer=fun_control["writer"],
    writerId="model_spot_cv",
    device = fun_control["device"])
Fold: 1
Epoch: 1
Loss on hold-out set: 2.3965971734788685
Accuracy on hold-out set: 0.14084507042253522
MAPK value on hold-out data: 0.2361110895872116
Epoch: 2
Loss on hold-out set: 2.39389388428794
Accuracy on hold-out set: 0.11267605633802817
MAPK value on hold-out data: 0.2083333134651184
Epoch: 3
Loss on hold-out set: 2.3877692090140448
Accuracy on hold-out set: 0.11267605633802817
MAPK value on hold-out data: 0.21064813435077667
Epoch: 4
Loss on hold-out set: 2.378531734148661
Accuracy on hold-out set: 0.2112676056338028
MAPK value on hold-out data: 0.3263889253139496
Fold: 2
Epoch: 1
Loss on hold-out set: 2.395424723625183
Accuracy on hold-out set: 0.15492957746478872
MAPK value on hold-out data: 0.23842591047286987
Epoch: 2
Loss on hold-out set: 2.3908526831203036
Accuracy on hold-out set: 0.11267605633802817
MAPK value on hold-out data: 0.2291666716337204
Epoch: 3
Loss on hold-out set: 2.3855505916807385
Accuracy on hold-out set: 0.11267605633802817
MAPK value on hold-out data: 0.2361111044883728
Epoch: 4
Loss on hold-out set: 2.3781197402212353
Accuracy on hold-out set: 0.2112676056338028
MAPK value on hold-out data: 0.31481486558914185
Fold: 3
Epoch: 1
Loss on hold-out set: 2.3950528701146445
Accuracy on hold-out set: 0.15492957746478872
MAPK value on hold-out data: 0.27314814925193787
Epoch: 2
Loss on hold-out set: 2.3885232739978366
Accuracy on hold-out set: 0.15492957746478872
MAPK value on hold-out data: 0.26851850748062134
Epoch: 3
Loss on hold-out set: 2.378796762890286
Accuracy on hold-out set: 0.18309859154929578
MAPK value on hold-out data: 0.2708333730697632
Epoch: 4
Loss on hold-out set: 2.369145327144199
Accuracy on hold-out set: 0.22535211267605634
MAPK value on hold-out data: 0.3402777910232544
Fold: 4
Epoch: 1
Loss on hold-out set: 2.3966386251979404
Accuracy on hold-out set: 0.1267605633802817
MAPK value on hold-out data: 0.2222221940755844
Epoch: 2
Loss on hold-out set: 2.3935044474071927
Accuracy on hold-out set: 0.1267605633802817
MAPK value on hold-out data: 0.2152777463197708
Epoch: 3
Loss on hold-out set: 2.389048437277476
Accuracy on hold-out set: 0.1267605633802817
MAPK value on hold-out data: 0.2291666716337204
Epoch: 4
Loss on hold-out set: 2.383568604787191
Accuracy on hold-out set: 0.15492957746478872
MAPK value on hold-out data: 0.2685185372829437
Fold: 5
Epoch: 1
Loss on hold-out set: 2.3971670402420893
Accuracy on hold-out set: 0.1267605633802817
MAPK value on hold-out data: 0.18518517911434174
Epoch: 2
Loss on hold-out set: 2.396760192182329
Accuracy on hold-out set: 0.028169014084507043
MAPK value on hold-out data: 0.12037035822868347
Epoch: 3
Loss on hold-out set: 2.395657433403863
Accuracy on hold-out set: 0.028169014084507043
MAPK value on hold-out data: 0.12962964177131653
Epoch: 4
Loss on hold-out set: 2.3930057088534036
Accuracy on hold-out set: 0.15492957746478872
MAPK value on hold-out data: 0.21990741789340973
Fold: 6
Epoch: 1
Loss on hold-out set: 2.3968306448724537
Accuracy on hold-out set: 0.09859154929577464
MAPK value on hold-out data: 0.2083333283662796
Epoch: 2
Loss on hold-out set: 2.394512951374054
Accuracy on hold-out set: 0.09859154929577464
MAPK value on hold-out data: 0.21296292543411255
Epoch: 3
Loss on hold-out set: 2.393150356080797
Accuracy on hold-out set: 0.09859154929577464
MAPK value on hold-out data: 0.21296294033527374
Epoch: 4
Loss on hold-out set: 2.3876595894495645
Accuracy on hold-out set: 0.09859154929577464
MAPK value on hold-out data: 0.24768519401550293
Fold: 7
Epoch: 1
Loss on hold-out set: 2.3964584867159524
Accuracy on hold-out set: 0.09859154929577464
MAPK value on hold-out data: 0.21990738809108734
Epoch: 2
Loss on hold-out set: 2.3930172986454434
Accuracy on hold-out set: 0.09859154929577464
MAPK value on hold-out data: 0.19212962687015533
Epoch: 3
Loss on hold-out set: 2.3863781690597534
Accuracy on hold-out set: 0.09859154929577464
MAPK value on hold-out data: 0.1875
Epoch: 4
Loss on hold-out set: 2.38067626953125
Accuracy on hold-out set: 0.14084507042253522
MAPK value on hold-out data: 0.24768519401550293
Fold: 8
Epoch: 1
Loss on hold-out set: 2.3971375737871443
Accuracy on hold-out set: 0.07142857142857142
MAPK value on hold-out data: 0.14523810148239136
Epoch: 2
Loss on hold-out set: 2.39499192237854
Accuracy on hold-out set: 0.07142857142857142
MAPK value on hold-out data: 0.17142857611179352
Epoch: 3
Loss on hold-out set: 2.3930067675454274
Accuracy on hold-out set: 0.07142857142857142
MAPK value on hold-out data: 0.15952380001544952
Epoch: 4
Loss on hold-out set: 2.386059808731079
Accuracy on hold-out set: 0.07142857142857142
MAPK value on hold-out data: 0.1904761791229248
Fold: 9
Epoch: 1
Loss on hold-out set: 2.396342536381313
Accuracy on hold-out set: 0.12857142857142856
MAPK value on hold-out data: 0.24047617614269257
Epoch: 2
Loss on hold-out set: 2.3934785774775915
Accuracy on hold-out set: 0.12857142857142856
MAPK value on hold-out data: 0.18809521198272705
Epoch: 3
Loss on hold-out set: 2.3868896211896624
Accuracy on hold-out set: 0.14285714285714285
MAPK value on hold-out data: 0.19761903584003448
Epoch: 4
Loss on hold-out set: 2.3813461031232563
Accuracy on hold-out set: 0.15714285714285714
MAPK value on hold-out data: 0.25238093733787537
Fold: 10
Epoch: 1
Loss on hold-out set: 2.396171685627529
Accuracy on hold-out set: 0.11428571428571428
MAPK value on hold-out data: 0.20714285969734192
Epoch: 2
Loss on hold-out set: 2.392170538221087
Accuracy on hold-out set: 0.1
MAPK value on hold-out data: 0.22380951046943665
Epoch: 3
Loss on hold-out set: 2.3834446702684673
Accuracy on hold-out set: 0.15714285714285714
MAPK value on hold-out data: 0.2666666805744171
Epoch: 4
Loss on hold-out set: 2.376386295046125
Accuracy on hold-out set: 0.15714285714285714
MAPK value on hold-out data: 0.3214285671710968
metric_name = type(fun_control["metric_torch"]).__name__
print(f"loss: {df_eval}, Cross-validated {metric_name}: {df_metrics}")
loss: 2.3814499181035966, Cross-validated MAPK: 0.27295634150505066

21.11 Detailed Hyperparameter Plots

filename = "./figures/" + experiment_name
spot_tuner.plot_important_hyperparameter_contour(filename=filename)
l1:  100.0
dropout_prob:  33.812420933242834
batch_size:  10.721845907868081
patience:  2.753841538123112
optimizer:  11.883811709818847

Contour plots.

21.12 Parallel Coordinates Plot

spot_tuner.parallel_plot()

Parallel coordinates plots

# close tensorbaoard writer
if fun_control["writer"] is not None:
    fun_control["writer"].close()

21.13 Plot all Combinations of Hyperparameters

  • Warning: this may take a while.
PLOT_ALL = False
if PLOT_ALL:
    n = spot_tuner.k
    for i in range(n-1):
        for j in range(i+1, n):
            spot_tuner.plot_contour(i=i, j=j, min_z=min_z, max_z = max_z)