INFO (n3fit.backends.keras_backend.internal_state): Clearing session
INFO (n3fit.model_trainer): Performing hyperparameter scan
INFO (n3fit.model_trainer):  > > Testing stopping_patience = 0.3600000000000001
INFO (n3fit.model_trainer):  > > Testing initializer = glorot_normal
INFO (n3fit.model_trainer):  > > Testing positivity = {'multiplier': 1.074400261320179}
INFO (n3fit.model_trainer):  > > Testing dropout = 0.15
INFO (n3fit.model_trainer):  > > Testing optimizer = {'learning_rate': 0.015380823956886622, 'optimizer_name': 'RMSprop'}
INFO (n3fit.model_trainer):  > > Testing activation_per_layer = sigmoid
INFO (n3fit.model_trainer):  > > Testing nodes_per_layer = (15, 41, 36, 45, 8)
INFO (n3fit.model_trainer):  > > Testing epochs = 35
INFO (n3fit.model_trainer): Generating layers
INFO (n3fit.model_trainer): Generating the input grid
INFO (n3fit.model_trainer): Generating the Model
INFO (n3fit.model_trainer): Fold 1 finished, loss=4.1, pass=True
INFO (n3fit.model_trainer): Generating the Model
WARNING (tensorflow): 5 out of the last 7 calls to <function TensorFlowTrainer.make_predict_function.<locals>.one_step_on_data_distributed at 0x7f8a9441c180> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
WARNING (tensorflow): 6 out of the last 8 calls to <function TensorFlowTrainer.make_predict_function.<locals>.one_step_on_data_distributed at 0x7f8a74609940> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
INFO (n3fit.model_trainer): Fold 2 finished, loss=7.6, pass=True
INFO (n3fit.backends.keras_backend.internal_state): Clearing session
INFO (n3fit.model_trainer): Performing hyperparameter scan
INFO (n3fit.model_trainer):  > > Testing optimizer = {'clipnorm': 1.8146481209154255, 'learning_rate': 0.06244684351854571, 'optimizer_name': 'Adadelta'}
INFO (n3fit.model_trainer):  > > Testing nodes_per_layer = (39, 48, 8)
INFO (n3fit.model_trainer):  > > Testing dropout = 0.12
INFO (n3fit.model_trainer):  > > Testing initializer = glorot_uniform
INFO (n3fit.model_trainer):  > > Testing stopping_patience = 0.12000000000000002
INFO (n3fit.model_trainer):  > > Testing epochs = 47
INFO (n3fit.model_trainer):  > > Testing positivity = {'multiplier': 1.0818441510036818}
INFO (n3fit.model_trainer):  > > Testing activation_per_layer = tanh
INFO (n3fit.model_trainer): Generating layers
INFO (n3fit.model_trainer): Generating the input grid
INFO (n3fit.model_trainer): Generating the Model
INFO (n3fit.model_trainer): Fold 1 finished, loss=23.4, pass=True
INFO (n3fit.model_trainer): Generating the Model
WARNING (tensorflow): 5 out of the last 7 calls to <function TensorFlowTrainer.make_predict_function.<locals>.one_step_on_data_distributed at 0x7f7090541f80> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
WARNING (tensorflow): 6 out of the last 8 calls to <function TensorFlowTrainer.make_predict_function.<locals>.one_step_on_data_distributed at 0x7f703ff3d120> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
INFO (n3fit.model_trainer): Fold 2 finished, loss=181.7, pass=True
INFO (n3fit.backends.keras_backend.internal_state): Clearing session
INFO (n3fit.model_trainer): Performing hyperparameter scan
INFO (n3fit.model_trainer):  > > Testing nodes_per_layer = (11, 15, 12, 12, 42, 8)
INFO (n3fit.model_trainer):  > > Testing epochs = 35
INFO (n3fit.model_trainer):  > > Testing stopping_patience = 0.24000000000000005
INFO (n3fit.model_trainer):  > > Testing positivity = {'multiplier': 1.0225601378667375}
INFO (n3fit.model_trainer):  > > Testing initializer = glorot_uniform
INFO (n3fit.model_trainer):  > > Testing optimizer = {'clipnorm': 1.9728045735692397, 'learning_rate': 0.06859128108496444, 'optimizer_name': 'Adadelta'}
INFO (n3fit.model_trainer):  > > Testing dropout = 0.06
INFO (n3fit.model_trainer):  > > Testing activation_per_layer = tanh
INFO (n3fit.model_trainer): Generating layers
INFO (n3fit.model_trainer): Generating the input grid
INFO (n3fit.model_trainer): Generating the Model
INFO (n3fit.model_trainer): Fold 1 finished, loss=7.7, pass=True
INFO (n3fit.model_trainer): Generating the Model
WARNING (tensorflow): 5 out of the last 7 calls to <function TensorFlowTrainer.make_predict_function.<locals>.one_step_on_data_distributed at 0x7f2fc06682c0> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
WARNING (tensorflow): 6 out of the last 8 calls to <function TensorFlowTrainer.make_predict_function.<locals>.one_step_on_data_distributed at 0x7f2fc058a840> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
INFO (n3fit.model_trainer): Fold 2 finished, loss=202.4, pass=True
INFO (n3fit.backends.keras_backend.internal_state): Clearing session
INFO (n3fit.model_trainer): Performing hyperparameter scan
INFO (n3fit.model_trainer):  > > Testing epochs = 35
INFO (n3fit.model_trainer):  > > Testing dropout = 0.15
INFO (n3fit.model_trainer):  > > Testing stopping_patience = 0.3600000000000001
INFO (n3fit.model_trainer):  > > Testing positivity = {'multiplier': 1.074400261320179}
INFO (n3fit.model_trainer):  > > Testing optimizer = {'learning_rate': 0.015380823956886622, 'optimizer_name': 'RMSprop'}
INFO (n3fit.model_trainer):  > > Testing nodes_per_layer = (15, 41, 36, 45, 8)
INFO (n3fit.model_trainer):  > > Testing activation_per_layer = sigmoid
INFO (n3fit.model_trainer):  > > Testing initializer = glorot_normal
INFO (n3fit.model_trainer): Generating layers
INFO (n3fit.model_trainer): Generating the input grid
INFO (n3fit.model_trainer): Generating the Model
INFO (n3fit.model_trainer): Fold 1 finished, loss=3.9, pass=True
INFO (n3fit.model_trainer): Generating the Model
WARNING (tensorflow): 5 out of the last 7 calls to <function TensorFlowTrainer.make_predict_function.<locals>.one_step_on_data_distributed at 0x7f9d045bc180> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
WARNING (tensorflow): 6 out of the last 8 calls to <function TensorFlowTrainer.make_predict_function.<locals>.one_step_on_data_distributed at 0x7f9cfe7b3560> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
INFO (n3fit.model_trainer): Fold 2 finished, loss=5.1, pass=True
INFO (n3fit.backends.keras_backend.internal_state): Clearing session
INFO (n3fit.model_trainer): Performing hyperparameter scan
INFO (n3fit.model_trainer):  > > Testing initializer = glorot_uniform
INFO (n3fit.model_trainer):  > > Testing stopping_patience = 0.12000000000000002
INFO (n3fit.model_trainer):  > > Testing optimizer = {'clipnorm': 1.8146481209154255, 'learning_rate': 0.06244684351854571, 'optimizer_name': 'Adadelta'}
INFO (n3fit.model_trainer):  > > Testing nodes_per_layer = (39, 48, 8)
INFO (n3fit.model_trainer):  > > Testing dropout = 0.12
INFO (n3fit.model_trainer):  > > Testing epochs = 47
INFO (n3fit.model_trainer):  > > Testing positivity = {'multiplier': 1.0818441510036818}
INFO (n3fit.model_trainer):  > > Testing activation_per_layer = tanh
INFO (n3fit.model_trainer): Generating layers
INFO (n3fit.model_trainer): Generating the input grid
INFO (n3fit.model_trainer): Generating the Model
INFO (n3fit.model_trainer): Fold 1 finished, loss=23.9, pass=True
INFO (n3fit.model_trainer): Generating the Model
WARNING (tensorflow): 5 out of the last 7 calls to <function TensorFlowTrainer.make_predict_function.<locals>.one_step_on_data_distributed at 0x7f6be6f2df80> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
WARNING (tensorflow): 6 out of the last 8 calls to <function TensorFlowTrainer.make_predict_function.<locals>.one_step_on_data_distributed at 0x7f6be4c2d120> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
INFO (n3fit.model_trainer): Fold 2 finished, loss=183.1, pass=True
INFO (n3fit.backends.keras_backend.internal_state): Clearing session
INFO (n3fit.model_trainer): Performing hyperparameter scan
INFO (n3fit.model_trainer):  > > Testing initializer = glorot_uniform
INFO (n3fit.model_trainer):  > > Testing positivity = {'multiplier': 1.0535710986795608}
INFO (n3fit.model_trainer):  > > Testing stopping_patience = 0.3600000000000001
INFO (n3fit.model_trainer):  > > Testing optimizer = {'clipnorm': 0.6864033680934294, 'learning_rate': 0.06354115109731578, 'optimizer_name': 'Adadelta'}
INFO (n3fit.model_trainer):  > > Testing nodes_per_layer = (17, 43, 37, 30, 37, 8)
INFO (n3fit.model_trainer):  > > Testing epochs = 36
INFO (n3fit.model_trainer):  > > Testing activation_per_layer = sigmoid
INFO (n3fit.model_trainer):  > > Testing dropout = 0.0
INFO (n3fit.model_trainer): Generating layers
INFO (n3fit.model_trainer): Generating the input grid
INFO (n3fit.model_trainer): Generating the Model
INFO (n3fit.model_trainer): Fold 1 finished, loss=3.6, pass=True
INFO (n3fit.model_trainer): Generating the Model
WARNING (tensorflow): 5 out of the last 7 calls to <function TensorFlowTrainer.make_predict_function.<locals>.one_step_on_data_distributed at 0x7f7188709580> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
WARNING (tensorflow): 6 out of the last 8 calls to <function TensorFlowTrainer.make_predict_function.<locals>.one_step_on_data_distributed at 0x7f7189e86de0> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
INFO (n3fit.model_trainer): Fold 2 finished, loss=153.2, pass=True
INFO (n3fit.backends.keras_backend.internal_state): Clearing session
INFO (n3fit.model_trainer): Performing hyperparameter scan
INFO (n3fit.model_trainer):  > > Testing dropout = 0.09
INFO (n3fit.model_trainer):  > > Testing positivity = {'multiplier': 1.0154032048444168}
INFO (n3fit.model_trainer):  > > Testing epochs = 49
INFO (n3fit.model_trainer):  > > Testing stopping_patience = 0.24000000000000005
INFO (n3fit.model_trainer):  > > Testing activation_per_layer = tanh
INFO (n3fit.model_trainer):  > > Testing nodes_per_layer = (29, 44, 8)
INFO (n3fit.model_trainer):  > > Testing initializer = glorot_normal
INFO (n3fit.model_trainer):  > > Testing optimizer = {'clipnorm': 0.7428693798462024, 'learning_rate': 0.06774487395290235, 'optimizer_name': 'Adadelta'}
INFO (n3fit.model_trainer): Generating layers
INFO (n3fit.model_trainer): Generating the input grid
INFO (n3fit.model_trainer): Generating the Model
INFO (n3fit.model_trainer): Fold 1 finished, loss=17.4, pass=True
INFO (n3fit.model_trainer): Generating the Model
WARNING (tensorflow): 5 out of the last 7 calls to <function TensorFlowTrainer.make_predict_function.<locals>.one_step_on_data_distributed at 0x7f2700721300> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
WARNING (tensorflow): 6 out of the last 8 calls to <function TensorFlowTrainer.make_predict_function.<locals>.one_step_on_data_distributed at 0x7f2700225120> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
INFO (n3fit.model_trainer): Fold 2 finished, loss=183.8, pass=True
INFO (n3fit.backends.keras_backend.internal_state): Clearing session
INFO (n3fit.model_trainer): Performing hyperparameter scan
INFO (n3fit.model_trainer):  > > Testing nodes_per_layer = (24, 22, 32, 8)
INFO (n3fit.model_trainer):  > > Testing stopping_patience = 0.30000000000000004
INFO (n3fit.model_trainer):  > > Testing activation_per_layer = sigmoid
INFO (n3fit.model_trainer):  > > Testing epochs = 48
INFO (n3fit.model_trainer):  > > Testing optimizer = {'learning_rate': 0.038818571810259626, 'optimizer_name': 'RMSprop'}
INFO (n3fit.model_trainer):  > > Testing positivity = {'multiplier': 1.0936628855044628}
INFO (n3fit.model_trainer):  > > Testing initializer = glorot_uniform
INFO (n3fit.model_trainer):  > > Testing dropout = 0.09
INFO (n3fit.model_trainer): Generating layers
INFO (n3fit.model_trainer): Generating the input grid
INFO (n3fit.model_trainer): Generating the Model
INFO (n3fit.model_trainer): Fold 1 finished, loss=5.4, pass=True
INFO (n3fit.model_trainer): Generating the Model
WARNING (tensorflow): 5 out of the last 7 calls to <function TensorFlowTrainer.make_predict_function.<locals>.one_step_on_data_distributed at 0x7f8780209c60> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
WARNING (tensorflow): 6 out of the last 8 calls to <function TensorFlowTrainer.make_predict_function.<locals>.one_step_on_data_distributed at 0x7f87785f7c40> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
INFO (n3fit.model_trainer): Fold 2 finished, loss=7.3, pass=True
INFO (n3fit.backends.keras_backend.internal_state): Clearing session
INFO (n3fit.model_trainer): Performing hyperparameter scan
INFO (n3fit.model_trainer):  > > Testing nodes_per_layer = (50, 35, 29, 8)
INFO (n3fit.model_trainer):  > > Testing epochs = 23
INFO (n3fit.model_trainer):  > > Testing dropout = 0.0
INFO (n3fit.model_trainer):  > > Testing stopping_patience = 0.12000000000000002
INFO (n3fit.model_trainer):  > > Testing activation_per_layer = sigmoid
INFO (n3fit.model_trainer):  > > Testing optimizer = {'clipnorm': 1.1070088671371212, 'learning_rate': 0.016177043880882706, 'optimizer_name': 'Adadelta'}
INFO (n3fit.model_trainer):  > > Testing initializer = glorot_normal
INFO (n3fit.model_trainer):  > > Testing positivity = {'multiplier': 1.0410293527702728}
INFO (n3fit.model_trainer): Generating layers
INFO (n3fit.model_trainer): Generating the input grid
INFO (n3fit.model_trainer): Generating the Model
INFO (n3fit.model_trainer): Fold 1 finished, loss=10.2, pass=True
INFO (n3fit.model_trainer): Generating the Model
WARNING (tensorflow): 5 out of the last 7 calls to <function TensorFlowTrainer.make_predict_function.<locals>.one_step_on_data_distributed at 0x7f99647f1da0> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
WARNING (tensorflow): 6 out of the last 8 calls to <function TensorFlowTrainer.make_predict_function.<locals>.one_step_on_data_distributed at 0x7f9964141940> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
INFO (n3fit.model_trainer): Fold 2 finished, loss=351.7, pass=True
INFO (n3fit.backends.keras_backend.internal_state): Clearing session
INFO (n3fit.model_trainer): Performing hyperparameter scan
INFO (n3fit.model_trainer):  > > Testing stopping_patience = 0.30000000000000004
INFO (n3fit.model_trainer):  > > Testing positivity = {'multiplier': 1.0221965660612466}
INFO (n3fit.model_trainer):  > > Testing nodes_per_layer = (28, 40, 16, 8)
INFO (n3fit.model_trainer):  > > Testing dropout = 0.03
INFO (n3fit.model_trainer):  > > Testing initializer = glorot_normal
INFO (n3fit.model_trainer):  > > Testing activation_per_layer = tanh
INFO (n3fit.model_trainer):  > > Testing optimizer = {'clipnorm': 1.9503952040053743, 'learning_rate': 0.01037098743852159, 'optimizer_name': 'Adadelta'}
INFO (n3fit.model_trainer):  > > Testing epochs = 24
INFO (n3fit.model_trainer): Generating layers
INFO (n3fit.model_trainer): Generating the input grid
INFO (n3fit.model_trainer): Generating the Model
INFO (n3fit.model_trainer): Fold 1 finished, loss=9.4, pass=True
INFO (n3fit.model_trainer): Generating the Model
WARNING (tensorflow): 5 out of the last 7 calls to <function TensorFlowTrainer.make_predict_function.<locals>.one_step_on_data_distributed at 0x7f97d8744040> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
WARNING (tensorflow): 6 out of the last 8 calls to <function TensorFlowTrainer.make_predict_function.<locals>.one_step_on_data_distributed at 0x7f97d9f74d60> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
INFO (n3fit.model_trainer): Fold 2 finished, loss=177.2, pass=True
INFO (n3fit.backends.keras_backend.internal_state): Clearing session
INFO (n3fit.model_trainer): Performing hyperparameter scan
INFO (n3fit.model_trainer):  > > Testing dropout = 0.09
INFO (n3fit.model_trainer):  > > Testing initializer = glorot_uniform
INFO (n3fit.model_trainer):  > > Testing nodes_per_layer = (16, 35, 8)
INFO (n3fit.model_trainer):  > > Testing stopping_patience = 0.12000000000000002
INFO (n3fit.model_trainer):  > > Testing positivity = {'multiplier': 1.0615455307107098}
INFO (n3fit.model_trainer):  > > Testing activation_per_layer = tanh
INFO (n3fit.model_trainer):  > > Testing optimizer = {'clipnorm': 0.8411342478713798, 'learning_rate': 0.04928810632634438, 'optimizer_name': 'Adadelta'}
INFO (n3fit.model_trainer):  > > Testing epochs = 47
INFO (n3fit.model_trainer): Generating layers
INFO (n3fit.model_trainer): Generating the input grid
INFO (n3fit.model_trainer): Generating the Model
INFO (n3fit.model_trainer): Fold 1 finished, loss=16.9, pass=True
INFO (n3fit.model_trainer): Generating the Model
WARNING (tensorflow): 5 out of the last 7 calls to <function TensorFlowTrainer.make_predict_function.<locals>.one_step_on_data_distributed at 0x7f907c2c60c0> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
WARNING (tensorflow): 6 out of the last 8 calls to <function TensorFlowTrainer.make_predict_function.<locals>.one_step_on_data_distributed at 0x7f905c471120> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
INFO (n3fit.model_trainer): Fold 2 finished, loss=399.5, pass=True
INFO (n3fit.backends.keras_backend.internal_state): Clearing session
INFO (n3fit.model_trainer): Performing hyperparameter scan
INFO (n3fit.model_trainer):  > > Testing optimizer = {'clipnorm': 1.7558937825962389, 'learning_rate': 0.02971486397602543, 'optimizer_name': 'Adadelta'}
INFO (n3fit.model_trainer):  > > Testing dropout = 0.03
INFO (n3fit.model_trainer):  > > Testing nodes_per_layer = (13, 33, 12, 44, 8)
INFO (n3fit.model_trainer):  > > Testing positivity = {'multiplier': 1.0896393776712885}
INFO (n3fit.model_trainer):  > > Testing activation_per_layer = sigmoid
INFO (n3fit.model_trainer):  > > Testing initializer = glorot_normal
INFO (n3fit.model_trainer):  > > Testing stopping_patience = 0.18000000000000005
INFO (n3fit.model_trainer):  > > Testing epochs = 30
INFO (n3fit.model_trainer): Generating layers
INFO (n3fit.model_trainer): Generating the input grid
INFO (n3fit.model_trainer): Generating the Model
INFO (n3fit.model_trainer): Fold 1 finished, loss=20.9, pass=True
INFO (n3fit.model_trainer): Generating the Model
WARNING (tensorflow): 5 out of the last 7 calls to <function TensorFlowTrainer.make_predict_function.<locals>.one_step_on_data_distributed at 0x7f15d44191c0> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
WARNING (tensorflow): 6 out of the last 8 calls to <function TensorFlowTrainer.make_predict_function.<locals>.one_step_on_data_distributed at 0x7f159d50e520> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
INFO (n3fit.model_trainer): Fold 2 finished, loss=149.7, pass=True
INFO (n3fit.backends.keras_backend.internal_state): Clearing session
INFO (n3fit.model_trainer): Performing hyperparameter scan
INFO (n3fit.model_trainer):  > > Testing optimizer = {'learning_rate': 0.010316672308836034, 'optimizer_name': 'RMSprop'}
INFO (n3fit.model_trainer):  > > Testing initializer = glorot_normal
INFO (n3fit.model_trainer):  > > Testing activation_per_layer = sigmoid
INFO (n3fit.model_trainer):  > > Testing dropout = 0.15
INFO (n3fit.model_trainer):  > > Testing nodes_per_layer = (36, 47, 50, 19, 8)
INFO (n3fit.model_trainer):  > > Testing stopping_patience = 0.4200000000000001
INFO (n3fit.model_trainer):  > > Testing epochs = 40
INFO (n3fit.model_trainer):  > > Testing positivity = {'multiplier': 1.066710317179611}
INFO (n3fit.model_trainer): Generating layers
INFO (n3fit.model_trainer): Generating the input grid
INFO (n3fit.model_trainer): Generating the Model
INFO (n3fit.model_trainer): Fold 1 finished, loss=2.9, pass=True
INFO (n3fit.model_trainer): Generating the Model
WARNING (tensorflow): 5 out of the last 7 calls to <function TensorFlowTrainer.make_predict_function.<locals>.one_step_on_data_distributed at 0x7fda081b4180> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
WARNING (tensorflow): 6 out of the last 8 calls to <function TensorFlowTrainer.make_predict_function.<locals>.one_step_on_data_distributed at 0x7fd9e02ab600> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
INFO (n3fit.model_trainer): Fold 2 finished, loss=11.7, pass=True
INFO (n3fit.backends.keras_backend.internal_state): Clearing session
INFO (n3fit.model_trainer): Performing hyperparameter scan
INFO (n3fit.model_trainer):  > > Testing stopping_patience = 0.3600000000000001
INFO (n3fit.model_trainer):  > > Testing nodes_per_layer = (48, 45, 44, 33, 8)
INFO (n3fit.model_trainer):  > > Testing epochs = 27
INFO (n3fit.model_trainer):  > > Testing optimizer = {'learning_rate': 0.09044437523236702, 'optimizer_name': 'RMSprop'}
INFO (n3fit.model_trainer):  > > Testing activation_per_layer = sigmoid
INFO (n3fit.model_trainer):  > > Testing initializer = glorot_normal
INFO (n3fit.model_trainer):  > > Testing positivity = {'multiplier': 1.0487923653696274}
INFO (n3fit.model_trainer):  > > Testing dropout = 0.12
INFO (n3fit.model_trainer): Generating layers
INFO (n3fit.model_trainer): Generating the input grid
INFO (n3fit.model_trainer): Generating the Model
INFO (n3fit.model_trainer): Fold 1 finished, loss=1.1, pass=True
INFO (n3fit.model_trainer): Generating the Model
WARNING (tensorflow): 5 out of the last 7 calls to <function TensorFlowTrainer.make_predict_function.<locals>.one_step_on_data_distributed at 0x7efd2c6f0fe0> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
WARNING (tensorflow): 6 out of the last 8 calls to <function TensorFlowTrainer.make_predict_function.<locals>.one_step_on_data_distributed at 0x7efd2c1de700> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
INFO (n3fit.model_trainer): Fold 2 finished, loss=6.0, pass=True
