Building NARX Neural Network using Sysidentpy

Example created by Wilson Rocha Lacerda Junior

Series-Parallen Training and Parallel prediction

Currently SysIdentPy support a Series-Parallel (open-loop) Feedforward Network training process, which make the training process easier. We convert the NARX network from Series-Parallel to the Parallel (closed-loop) configuration for prediction.

Series-Parallel allows us to use Pytorch directly for training, so we can use all the power of the Pytorch library to build our NARX Neural Network model!

The reader is referred to the following paper for a more in depth discussion about Series-Parallel and Parallel configurations regarding NARX neural network:

Parallel Training Considered Harmful?: Comparing series-parallel and parallel feedforward network training

Building a NARX Neural Network

First, just import the necessary packages

pip install sysidentpy
Requirement already satisfied: sysidentpy in c:\users\wilso\miniconda3\envs\sysidentpy\lib\site-packages (0.1.6)
Requirement already satisfied: matplotlib>=3.3.2 in c:\users\wilso\miniconda3\envs\sysidentpy\lib\site-packages (from sysidentpy) (3.4.3)
Requirement already satisfied: scipy>=1.7.0 in c:\users\wilso\miniconda3\envs\sysidentpy\lib\site-packages (from sysidentpy) (1.7.1)
Requirement already satisfied: numpy>=1.19.2 in c:\users\wilso\miniconda3\envs\sysidentpy\lib\site-packages (from sysidentpy) (1.20.3)
Requirement already satisfied: pyparsing>=2.2.1 in c:\users\wilso\miniconda3\envs\sysidentpy\lib\site-packages (from matplotlib>=3.3.2->sysidentpy) (2.4.7)
Requirement already satisfied: pillow>=6.2.0 in c:\users\wilso\miniconda3\envs\sysidentpy\lib\site-packages (from matplotlib>=3.3.2->sysidentpy) (8.3.2)
Requirement already satisfied: cycler>=0.10 in c:\users\wilso\miniconda3\envs\sysidentpy\lib\site-packages (from matplotlib>=3.3.2->sysidentpy) (0.10.0)
Requirement already satisfied: kiwisolver>=1.0.1 in c:\users\wilso\miniconda3\envs\sysidentpy\lib\site-packages (from matplotlib>=3.3.2->sysidentpy) (1.3.2)
Requirement already satisfied: python-dateutil>=2.7 in c:\users\wilso\miniconda3\envs\sysidentpy\lib\site-packages (from matplotlib>=3.3.2->sysidentpy) (2.8.2)
Requirement already satisfied: six in c:\users\wilso\miniconda3\envs\sysidentpy\lib\site-packages (from cycler>=0.10->matplotlib>=3.3.2->sysidentpy) (1.16.0)
Note: you may need to restart the kernel to use updated packages.
from torch import nn
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sysidentpy.metrics import mean_squared_error
from sysidentpy.utils.generate_data import get_siso_data
from sysidentpy.neural_network import NARXNN

Getting the data

The data is generated by simulating the following model:

\(y_k = 0.2y_{k-1} + 0.1y_{k-1}x_{k-1} + 0.9x_{k-1} + e_{k}\).

If colored_noise is set to True:

\(e_{k} = 0.8\nu_{k-1} + \nu_{k}\),

where \(x\) is a uniformly distributed random variable and \(\nu\) is a gaussian distributed variable with \(\mu=0\) and \(\sigma=0.1\)

x_train, x_valid, y_train, y_valid = get_siso_data(n=1000,
                                                   colored_noise=False,
                                                   sigma=0.01,
                                                   train_percentage=80)

Choosing the NARX parameters, loss function and optmizer

One can create a NARXNN object and choose the maximum lag of both input and output for building the regressor matrix to serve as input of the network.

In addition, you can choose the loss function, the optmizer, the optinional parameters of the optmizer, the number of epochs.

Because we built this feature on top of Pytorch, you can choose any of the loss function of the torch.nn.functional. Click here for a list of the loss functions you can use. You just need to pass the name of the loss function you want.

Similarly, you can choose any of the optimizers of the torch.optim. Click here for a list of optimizers available.

narx_net = NARXNN(ylag=2,
                  xlag=2,
                  loss_func='mse_loss',
                  optimizer='Adam',
                  epochs=200,
                  verbose=True,
                  optim_params={'betas': (0.9, 0.999), 'eps': 1e-05} # optional parameters of the optimizer
)

Since we have defined our NARXNN using \(ylag=2\) and \(xlag=2\), we have a regressor matrix with 4 features. We need the size of the regressor matrix to build the layers of our network. Our input data(x_train) have only one feature, but since we are creating an NARX network, a regressor matrix is built behind the scenes with new features based on the xlag and ylag.

You can check the size the of the regressor matrix by using the following line of code:

len(narx_net.regressor_code) # the number of features of the NARX net
4

Building the NARX Neural Network

The configuration of your network follows exactly the same pattern of a network defined in Pytorch. The following representing our NARX neural network.

class NARX(nn.Module):
    def __init__(self):
        super().__init__()
        self.lin = nn.Linear(4, 10)
        self.lin2 = nn.Linear(10, 10)
        self.lin3 = nn.Linear(10, 1)
        self.tanh = nn.Tanh()

    def forward(self, xb):
        z = self.lin(xb)
        z = self.tanh(z)
        z = self.lin2(z)
        z = self.tanh(z)
        z = self.lin3(z)
        return z

We have to pass the defined network to our NARXNN estimator.

narx_net.net = NARX() 

Numpy array to Tensor

If your inputs are numpy array, you can transform them to tensor using our data_transform method. Our function return a Dataloader object that we use to iterate on our training function. You can transform your inputs mannualy using TensorDataset and Dataloader using Pytorch.

Since our inputs are numpy arrays, we transform them using the following:

train_dl = narx_net.data_transform(x_train, y_train)
valid_dl = narx_net.data_transform(x_valid, y_valid)

Fit and Predict

Because we have a fit (for training) and predict function for Polynomial NARMAX, we create the same pattern for the NARX net. So, you only have to fit and predict using the following:

narx_net.fit(train_dl, valid_dl)
yhat = narx_net.predict(x_valid, y_valid)
10-16 17:14:04 - INFO - Train metrics: 0.13587702058237 | Validation metrics: 0.13731206185889966
10-16 17:14:04 - INFO - Train metrics: 0.03290778055394951 | Validation metrics: 0.03202500239466176
10-16 17:14:04 - INFO - Train metrics: 0.011123998088290668 | Validation metrics: 0.011168236402098578
10-16 17:14:04 - INFO - Train metrics: 0.00891545736102532 | Validation metrics: 0.00800760519323927
10-16 17:14:04 - INFO - Train metrics: 0.004802983189421825 | Validation metrics: 0.0043489806862039996
10-16 17:14:04 - INFO - Train metrics: 0.0034811852084365405 | Validation metrics: 0.0032854086785304424
10-16 17:14:04 - INFO - Train metrics: 0.002891695556956761 | Validation metrics: 0.0029569248706478664
10-16 17:14:04 - INFO - Train metrics: 0.0024196546744172437 | Validation metrics: 0.0023011573580930932
10-16 17:14:04 - INFO - Train metrics: 0.002234630874022001 | Validation metrics: 0.001972487658459806
10-16 17:14:04 - INFO - Train metrics: 0.002081210989350999 | Validation metrics: 0.0018769958282284665
10-16 17:14:04 - INFO - Train metrics: 0.001968406335996898 | Validation metrics: 0.0018830438276207207
10-16 17:14:04 - INFO - Train metrics: 0.0018524305380292628 | Validation metrics: 0.0017551232324536851
10-16 17:14:04 - INFO - Train metrics: 0.0017873112682397839 | Validation metrics: 0.0016568273275319224
10-16 17:14:04 - INFO - Train metrics: 0.0017135833415732648 | Validation metrics: 0.0016065942302270972
10-16 17:14:04 - INFO - Train metrics: 0.0016330079733391752 | Validation metrics: 0.001550019471765016
10-16 17:14:04 - INFO - Train metrics: 0.001573641862682905 | Validation metrics: 0.0014960551258138936
10-16 17:14:04 - INFO - Train metrics: 0.0015186268452877052 | Validation metrics: 0.0014426777482672473
10-16 17:14:04 - INFO - Train metrics: 0.0014675419798347268 | Validation metrics: 0.0014008096045334682
10-16 17:14:04 - INFO - Train metrics: 0.0014140893863327335 | Validation metrics: 0.0013571933224665546
10-16 17:14:04 - INFO - Train metrics: 0.0013636798611877108 | Validation metrics: 0.0013119126481916567
10-16 17:14:04 - INFO - Train metrics: 0.0013178434034162912 | Validation metrics: 0.0012713190182253268
10-16 17:14:04 - INFO - Train metrics: 0.0012756684137961096 | Validation metrics: 0.001234932513343114
10-16 17:14:04 - INFO - Train metrics: 0.0012357673125766347 | Validation metrics: 0.001200321741928958
10-16 17:14:04 - INFO - Train metrics: 0.001198016226415366 | Validation metrics: 0.0011668316996421176
10-16 17:14:04 - INFO - Train metrics: 0.001162860446398317 | Validation metrics: 0.0011356044854648939
10-16 17:14:04 - INFO - Train metrics: 0.0011302459857223514 | Validation metrics: 0.0011067499401932112
10-16 17:14:04 - INFO - Train metrics: 0.0011000164497391622 | Validation metrics: 0.0010794678554785523
10-16 17:14:04 - INFO - Train metrics: 0.001071757161674699 | Validation metrics: 0.0010536881046125082
10-16 17:14:04 - INFO - Train metrics: 0.0010443430497220024 | Validation metrics: 0.0010283222713630008
10-16 17:14:04 - INFO - Train metrics: 0.001016166659870318 | Validation metrics: 0.0010016743114895442
10-16 17:14:04 - INFO - Train metrics: 0.0009850776982773795 | Validation metrics: 0.0009717297260508393
10-16 17:14:04 - INFO - Train metrics: 0.000948687890319289 | Validation metrics: 0.0009363074892793189
10-16 17:14:04 - INFO - Train metrics: 0.0009049508343672468 | Validation metrics: 0.0008935367052368067
10-16 17:14:04 - INFO - Train metrics: 0.0008530269396190244 | Validation metrics: 0.0008428937013524424
10-16 17:14:04 - INFO - Train metrics: 0.000794030313979937 | Validation metrics: 0.0007857977344438132
10-16 17:14:04 - INFO - Train metrics: 0.0007313873232540704 | Validation metrics: 0.0007259279814069018
10-16 17:14:04 - INFO - Train metrics: 0.0006703230029965207 | Validation metrics: 0.0006686424450817132
10-16 17:14:04 - INFO - Train metrics: 0.0006164205753662271 | Validation metrics: 0.0006194670709329798
10-16 17:14:04 - INFO - Train metrics: 0.0005736945717499818 | Validation metrics: 0.000582134033637968
10-16 17:14:04 - INFO - Train metrics: 0.0005429866348065314 | Validation metrics: 0.0005570127578179418
10-16 17:14:04 - INFO - Train metrics: 0.0005216128828166225 | Validation metrics: 0.0005408442212325153
10-16 17:14:04 - INFO - Train metrics: 0.0005046903605212254 | Validation metrics: 0.0005282093682608595
10-16 17:14:04 - INFO - Train metrics: 0.00048766853763397436 | Validation metrics: 0.000514221653128231
10-16 17:14:04 - INFO - Train metrics: 0.00046855724936895033 | Validation metrics: 0.0004968927571117276
10-16 17:14:04 - INFO - Train metrics: 0.0004482132886427882 | Validation metrics: 0.00047744847122683294
10-16 17:14:04 - INFO - Train metrics: 0.00042862999926958427 | Validation metrics: 0.0004584821078672328
10-16 17:14:04 - INFO - Train metrics: 0.0004111042325437433 | Validation metrics: 0.00044180895054872813
10-16 17:14:04 - INFO - Train metrics: 0.00039572022122361306 | Validation metrics: 0.00042764867998123395
10-16 17:14:04 - INFO - Train metrics: 0.0003817325943286174 | Validation metrics: 0.0004149979066263621
10-16 17:14:04 - INFO - Train metrics: 0.0003683525283487145 | Validation metrics: 0.000402753748766363
10-16 17:14:04 - INFO - Train metrics: 0.00035548621062411084 | Validation metrics: 0.0003907927259040827
10-16 17:14:04 - INFO - Train metrics: 0.0003435152300101306 | Validation metrics: 0.0003796867912193064
10-16 17:14:04 - INFO - Train metrics: 0.00033253606701769253 | Validation metrics: 0.0003696229912086644
10-16 17:14:04 - INFO - Train metrics: 0.00032224440791079294 | Validation metrics: 0.00036019856942761123
10-16 17:14:04 - INFO - Train metrics: 0.00031246255870551377 | Validation metrics: 0.00035116862361037147
10-16 17:14:04 - INFO - Train metrics: 0.00030329204997442253 | Validation metrics: 0.0003426931601492782
10-16 17:14:04 - INFO - Train metrics: 0.00029473739270182716 | Validation metrics: 0.0003348093849108225
10-16 17:14:04 - INFO - Train metrics: 0.0002866539730177492 | Validation metrics: 0.0003273322842568347
10-16 17:14:04 - INFO - Train metrics: 0.00027901862288724847 | Validation metrics: 0.0003202354448442959
10-16 17:14:04 - INFO - Train metrics: 0.00027185551121797094 | Validation metrics: 0.00031356960170516614
10-16 17:14:04 - INFO - Train metrics: 0.00026510156833165253 | Validation metrics: 0.00030726640111286986
10-16 17:14:04 - INFO - Train metrics: 0.00025872202324745173 | Validation metrics: 0.00030128586263574586
10-16 17:14:05 - INFO - Train metrics: 0.0002527198453219654 | Validation metrics: 0.00029564098069312595
10-16 17:14:05 - INFO - Train metrics: 0.00024705800113447926 | Validation metrics: 0.00029029584968359105
10-16 17:14:05 - INFO - Train metrics: 0.00024171953223310738 | Validation metrics: 0.0002852367819874839
10-16 17:14:05 - INFO - Train metrics: 0.00023669169865312862 | Validation metrics: 0.0002804536380066602
10-16 17:14:05 - INFO - Train metrics: 0.00023195199959591628 | Validation metrics: 0.0002759257081609144
10-16 17:14:05 - INFO - Train metrics: 0.00022748933475841548 | Validation metrics: 0.0002716460686783786
10-16 17:14:05 - INFO - Train metrics: 0.00022328815489037565 | Validation metrics: 0.0002676006296303857
10-16 17:14:05 - INFO - Train metrics: 0.0002193364594394299 | Validation metrics: 0.0002637796632820891
10-16 17:14:05 - INFO - Train metrics: 0.00021562156559086002 | Validation metrics: 0.0002601730015194702
10-16 17:14:05 - INFO - Train metrics: 0.00021213452873096794 | Validation metrics: 0.00025677401704756034
10-16 17:14:05 - INFO - Train metrics: 0.00020886304289952182 | Validation metrics: 0.00025357183488203456
10-16 17:14:05 - INFO - Train metrics: 0.00020579978768080052 | Validation metrics: 0.0002505613944841304
10-16 17:14:05 - INFO - Train metrics: 0.0002029355050296287 | Validation metrics: 0.0002477344286021297
10-16 17:14:05 - INFO - Train metrics: 0.00020026439919899728 | Validation metrics: 0.00024508722943742994
10-16 17:14:05 - INFO - Train metrics: 0.00019777950262977627 | Validation metrics: 0.0002426143210602341
10-16 17:14:05 - INFO - Train metrics: 0.00019547522600062126 | Validation metrics: 0.0002403113764071028
10-16 17:14:05 - INFO - Train metrics: 0.00019334669916115627 | Validation metrics: 0.0002381743620987041
10-16 17:14:05 - INFO - Train metrics: 0.00019139115852397168 | Validation metrics: 0.00023620276308543227
10-16 17:14:05 - INFO - Train metrics: 0.000189605745312438 | Validation metrics: 0.00023439434366394774
10-16 17:14:05 - INFO - Train metrics: 0.00018798948338116404 | Validation metrics: 0.00023274924861845759
10-16 17:14:05 - INFO - Train metrics: 0.00018653914067318854 | Validation metrics: 0.00023126524709626053
10-16 17:14:05 - INFO - Train metrics: 0.0001852599527414907 | Validation metrics: 0.00022994916321065353
10-16 17:14:05 - INFO - Train metrics: 0.00018414963132475402 | Validation metrics: 0.00022879911285608707
10-16 17:14:05 - INFO - Train metrics: 0.00018321383773800881 | Validation metrics: 0.00022782251927319614
10-16 17:14:05 - INFO - Train metrics: 0.00018245647601575538 | Validation metrics: 0.00022702357517743503
10-16 17:14:05 - INFO - Train metrics: 0.0001818844420881475 | Validation metrics: 0.00022641034159489503
10-16 17:14:05 - INFO - Train metrics: 0.0001815058953125254 | Validation metrics: 0.00022599230181764472
10-16 17:14:05 - INFO - Train metrics: 0.00018133103169435473 | Validation metrics: 0.00022578031965884182
10-16 17:14:05 - INFO - Train metrics: 0.0001813711773284377 | Validation metrics: 0.00022578624464016445
10-16 17:14:05 - INFO - Train metrics: 0.0001816420637104394 | Validation metrics: 0.00022602712366885194
10-16 17:14:05 - INFO - Train metrics: 0.00018216038542481043 | Validation metrics: 0.00022652021002475962
10-16 17:14:05 - INFO - Train metrics: 0.0001829449652462502 | Validation metrics: 0.00022728428773163356
10-16 17:14:05 - INFO - Train metrics: 0.00018401651831844443 | Validation metrics: 0.0002283410048947171
10-16 17:14:05 - INFO - Train metrics: 0.00018540101080174724 | Validation metrics: 0.00022971687231076448
10-16 17:14:05 - INFO - Train metrics: 0.0001871255745415116 | Validation metrics: 0.00023143855658491527
10-16 17:14:05 - INFO - Train metrics: 0.00018921611391627615 | Validation metrics: 0.00023353199893138323
10-16 17:14:05 - INFO - Train metrics: 0.0001917025718804715 | Validation metrics: 0.00023602630523847173
10-16 17:14:05 - INFO - Train metrics: 0.00019461203392485347 | Validation metrics: 0.00023894694591449067
10-16 17:14:05 - INFO - Train metrics: 0.00019796993537487365 | Validation metrics: 0.00024231858704370832
10-16 17:14:05 - INFO - Train metrics: 0.0002017979500500163 | Validation metrics: 0.00024616048007502665
10-16 17:14:05 - INFO - Train metrics: 0.0002061067435558988 | Validation metrics: 0.0002504808984337036
10-16 17:14:05 - INFO - Train metrics: 0.00021089668640105947 | Validation metrics: 0.00025527590722306583
10-16 17:14:05 - INFO - Train metrics: 0.00021615068880166568 | Validation metrics: 0.0002605260176069075
10-16 17:14:05 - INFO - Train metrics: 0.0002218311248661689 | Validation metrics: 0.00026618850575007425
10-16 17:14:05 - INFO - Train metrics: 0.00022787108401304583 | Validation metrics: 0.00027219134290594455
10-16 17:14:05 - INFO - Train metrics: 0.00023417931752784744 | Validation metrics: 0.0002784403843565308
10-16 17:14:05 - INFO - Train metrics: 0.00024063166985574194 | Validation metrics: 0.000284805700313441
10-16 17:14:05 - INFO - Train metrics: 0.0002470695966403035 | Validation metrics: 0.00029112458992820007
10-16 17:14:05 - INFO - Train metrics: 0.0002533171978679353 | Validation metrics: 0.0002972187709258493
10-16 17:14:05 - INFO - Train metrics: 0.0002591841327368438 | Validation metrics: 0.00030289474281487103
10-16 17:14:05 - INFO - Train metrics: 0.0002644844667214018 | Validation metrics: 0.0003079659622421281
10-16 17:14:05 - INFO - Train metrics: 0.0002690459320244069 | Validation metrics: 0.0003122611810933008
10-16 17:14:05 - INFO - Train metrics: 0.0002727351051986913 | Validation metrics: 0.00031564751583518404
10-16 17:14:05 - INFO - Train metrics: 0.0002754697368499522 | Validation metrics: 0.00031804631454188077
10-16 17:14:05 - INFO - Train metrics: 0.00027723026697939367 | Validation metrics: 0.00031944153424131336
10-16 17:14:05 - INFO - Train metrics: 0.000278057908252136 | Validation metrics: 0.0003198790199339691
10-16 17:14:05 - INFO - Train metrics: 0.00027804844425541856 | Validation metrics: 0.00031945937841712036
10-16 17:14:05 - INFO - Train metrics: 0.0002773469772419185 | Validation metrics: 0.00031833314337424294
10-16 17:14:05 - INFO - Train metrics: 0.00027612252694112307 | Validation metrics: 0.0003166746173519641
10-16 17:14:05 - INFO - Train metrics: 0.0002745565689019485 | Validation metrics: 0.0003146692121817702
10-16 17:14:05 - INFO - Train metrics: 0.00027282492195362257 | Validation metrics: 0.0003124976129685011
10-16 17:14:05 - INFO - Train metrics: 0.00027108039291322294 | Validation metrics: 0.0003103167850481854
10-16 17:14:05 - INFO - Train metrics: 0.0002694476517585496 | Validation metrics: 0.0003082539852134733
10-16 17:14:05 - INFO - Train metrics: 0.00026802292038490504 | Validation metrics: 0.0003064072957248286
10-16 17:14:05 - INFO - Train metrics: 0.0002668703268215009 | Validation metrics: 0.0003048423377825229
10-16 17:14:05 - INFO - Train metrics: 0.00026601948151697537 | Validation metrics: 0.0003035889392850375
10-16 17:14:05 - INFO - Train metrics: 0.00026548119807612373 | Validation metrics: 0.00030265816365664995
10-16 17:14:05 - INFO - Train metrics: 0.00026523843952507145 | Validation metrics: 0.00030203251523257355
10-16 17:14:05 - INFO - Train metrics: 0.00026525962711364395 | Validation metrics: 0.0003016791620992613
10-16 17:14:05 - INFO - Train metrics: 0.00026550322146981227 | Validation metrics: 0.0003015555794447228
10-16 17:14:05 - INFO - Train metrics: 0.0002659198980927841 | Validation metrics: 0.0003016111074953876
10-16 17:14:05 - INFO - Train metrics: 0.0002664579625984299 | Validation metrics: 0.00030179336141428035
10-16 17:14:05 - INFO - Train metrics: 0.00026705964512330595 | Validation metrics: 0.0003020435754230923
10-16 17:14:05 - INFO - Train metrics: 0.0002676761823088948 | Validation metrics: 0.0003023123667890563
10-16 17:14:05 - INFO - Train metrics: 0.0002682543710766379 | Validation metrics: 0.0003025454730371653
10-16 17:14:05 - INFO - Train metrics: 0.0002687562160111642 | Validation metrics: 0.0003027050418431184
10-16 17:14:05 - INFO - Train metrics: 0.00026914922793758774 | Validation metrics: 0.0003027583816679044
10-16 17:14:05 - INFO - Train metrics: 0.0002694062223829525 | Validation metrics: 0.00030267827940112297
10-16 17:14:06 - INFO - Train metrics: 0.00026951231890157297 | Validation metrics: 0.00030245002261551114
10-16 17:14:06 - INFO - Train metrics: 0.00026946315337771917 | Validation metrics: 0.0003020696474575541
10-16 17:14:06 - INFO - Train metrics: 0.00026926281231013185 | Validation metrics: 0.00030154094065897014
10-16 17:14:06 - INFO - Train metrics: 0.0002689196518899518 | Validation metrics: 0.0003008730130415937
10-16 17:14:06 - INFO - Train metrics: 0.00026845371211035163 | Validation metrics: 0.0003000857807387777
10-16 17:14:06 - INFO - Train metrics: 0.0002678844650751003 | Validation metrics: 0.00029919948046697734
10-16 17:14:06 - INFO - Train metrics: 0.0002672360401573173 | Validation metrics: 0.00029823843803082715
10-16 17:14:06 - INFO - Train metrics: 0.0002665334869290522 | Validation metrics: 0.00029722661249729983
10-16 17:14:06 - INFO - Train metrics: 0.00026580459615659145 | Validation metrics: 0.00029619305742659954
10-16 17:14:06 - INFO - Train metrics: 0.00026507217729234263 | Validation metrics: 0.0002951600675287685
10-16 17:14:06 - INFO - Train metrics: 0.0002643524534241145 | Validation metrics: 0.0002941437908730498
10-16 17:14:06 - INFO - Train metrics: 0.00026366696988418 | Validation metrics: 0.00029316581294735463
10-16 17:14:06 - INFO - Train metrics: 0.0002630271729027017 | Validation metrics: 0.00029223688056156266
10-16 17:14:06 - INFO - Train metrics: 0.0002624423544428528 | Validation metrics: 0.0002913668105258804
10-16 17:14:06 - INFO - Train metrics: 0.00026191838961246337 | Validation metrics: 0.00029056087357340137
10-16 17:14:06 - INFO - Train metrics: 0.0002614544686831576 | Validation metrics: 0.00028981829786466225
10-16 17:14:06 - INFO - Train metrics: 0.0002610458724943566 | Validation metrics: 0.00028913417661143937
10-16 17:14:06 - INFO - Train metrics: 0.000260692460997553 | Validation metrics: 0.0002885081538062712
10-16 17:14:06 - INFO - Train metrics: 0.0002603890211788242 | Validation metrics: 0.0002879350167767361
10-16 17:14:06 - INFO - Train metrics: 0.00026012158925373124 | Validation metrics: 0.0002874008941668735
10-16 17:14:06 - INFO - Train metrics: 0.00025988189076256695 | Validation metrics: 0.00028689738599385953
10-16 17:14:06 - INFO - Train metrics: 0.0002596557131159659 | Validation metrics: 0.0002864100422058725
10-16 17:14:06 - INFO - Train metrics: 0.00025943661313433957 | Validation metrics: 0.00028593298265324775
10-16 17:14:06 - INFO - Train metrics: 0.0002592154970012703 | Validation metrics: 0.0002854570444800298
10-16 17:14:06 - INFO - Train metrics: 0.0002589802788952739 | Validation metrics: 0.0002849700771309812
10-16 17:14:06 - INFO - Train metrics: 0.00025872217959848075 | Validation metrics: 0.0002844638503956223
10-16 17:14:06 - INFO - Train metrics: 0.0002584342629167912 | Validation metrics: 0.00028393050315295994
10-16 17:14:06 - INFO - Train metrics: 0.0002581149959390549 | Validation metrics: 0.00028336990193694343
10-16 17:14:06 - INFO - Train metrics: 0.00025775641347935193 | Validation metrics: 0.00028277362501037055
10-16 17:14:06 - INFO - Train metrics: 0.00025735826301271736 | Validation metrics: 0.00028214176776913003
10-16 17:14:06 - INFO - Train metrics: 0.00025692011356175196 | Validation metrics: 0.0002814732135374645
10-16 17:14:06 - INFO - Train metrics: 0.0002564406798166902 | Validation metrics: 0.00028076779048518286
10-16 17:14:06 - INFO - Train metrics: 0.0002559227384215473 | Validation metrics: 0.0002800280585734531
10-16 17:14:06 - INFO - Train metrics: 0.0002553700370597945 | Validation metrics: 0.0002792572852217056
10-16 17:14:06 - INFO - Train metrics: 0.00025478379688876913 | Validation metrics: 0.00027845716536059183
10-16 17:14:06 - INFO - Train metrics: 0.0002541723697595672 | Validation metrics: 0.00027763578588627466
10-16 17:14:06 - INFO - Train metrics: 0.000253534810636531 | Validation metrics: 0.0002767925653101011
10-16 17:14:06 - INFO - Train metrics: 0.0002528810375434274 | Validation metrics: 0.00027593688623664544
10-16 17:14:06 - INFO - Train metrics: 0.00025221323651345654 | Validation metrics: 0.0002750710092103955
10-16 17:14:06 - INFO - Train metrics: 0.00025153078075660566 | Validation metrics: 0.00027419472403702035
10-16 17:14:06 - INFO - Train metrics: 0.0002508440031538738 | Validation metrics: 0.00027331727324053645
10-16 17:14:06 - INFO - Train metrics: 0.0002501562643588467 | Validation metrics: 0.0002724427639887753
10-16 17:14:06 - INFO - Train metrics: 0.000249466753074897 | Validation metrics: 0.00027157010665196325
10-16 17:14:06 - INFO - Train metrics: 0.00024877757072258496 | Validation metrics: 0.0002707017556531355
10-16 17:14:06 - INFO - Train metrics: 0.00024809029106054233 | Validation metrics: 0.0002698379473506727
10-16 17:14:06 - INFO - Train metrics: 0.0002474087150927918 | Validation metrics: 0.0002689837200879919
10-16 17:14:06 - INFO - Train metrics: 0.0002467290825169081 | Validation metrics: 0.00026813492656925297
10-16 17:14:06 - INFO - Train metrics: 0.0002460536062451929 | Validation metrics: 0.0002672932629010198
10-16 17:14:06 - INFO - Train metrics: 0.00024538291726182015 | Validation metrics: 0.0002664598628097788
10-16 17:14:06 - INFO - Train metrics: 0.00024471081790974116 | Validation metrics: 0.00026562877441521247
10-16 17:14:06 - INFO - Train metrics: 0.0002440385071719462 | Validation metrics: 0.0002648006296232155
10-16 17:14:06 - INFO - Train metrics: 0.0002433683684625007 | Validation metrics: 0.00026397751773604087
10-16 17:14:06 - INFO - Train metrics: 0.00024269327440650383 | Validation metrics: 0.00026315371470638746
10-16 17:14:06 - INFO - Train metrics: 0.00024201501047654185 | Validation metrics: 0.00026232958359719076
10-16 17:14:06 - INFO - Train metrics: 0.00024132890871727142 | Validation metrics: 0.000261501150559681
10-16 17:14:06 - INFO - Train metrics: 0.0002406337027108871 | Validation metrics: 0.00026066726202385104
10-16 17:14:06 - INFO - Train metrics: 0.0002399317935234015 | Validation metrics: 0.00025983011944459355
10-16 17:14:06 - INFO - Train metrics: 0.00023921692426891807 | Validation metrics: 0.0002589836477647733
10-16 17:14:06 - INFO - Train metrics: 0.00023848666857057403 | Validation metrics: 0.00025812561774856855
10-16 17:14:06 - INFO - Train metrics: 0.00023774105724633525 | Validation metrics: 0.00025725609171933333
print(mean_squared_error(y_valid, yhat))

ee, ex, extras, lam = narx_net.residuals(x_valid, y_valid, yhat)
narx_net.plot_result(y_valid, yhat, ee, ex)
0.0003313589478610249
../_images/narx_neural_network_20_11.png

Note

If you built the net configuration before calling the NARXNN, you can just pass the model to the NARXNN as follows:

class NARX(nn.Module):
    def __init__(self):
        super().__init__()
        self.lin = nn.Linear(4, 10)
        self.lin2 = nn.Linear(10, 10)
        self.lin3 = nn.Linear(10, 1)
        self.tanh = nn.Tanh()

    def forward(self, xb):
        z = self.lin(xb)
        z = self.tanh(z)
        z = self.lin2(z)
        z = self.tanh(z)
        z = self.lin3(z)
        return z

narx_net2 = NARXNN(net=NARX(),
                   ylag=2,
                   xlag=2,
                   loss_func='mse_loss',
                   optimizer='Adam',
                   epochs=200,
                   verbose=True,
                   optim_params={'betas': (0.9, 0.999), 'eps': 1e-05} # optional parameters of the optimizer
)

narx_net2.fit(train_dl, valid_dl)
yhat = narx_net2.predict(x_valid, y_valid)

ee, ex, extras, lam = narx_net2.residuals(x_valid, y_valid, yhat)
narx_net2.plot_result(y_valid, yhat, ee, ex)
10-16 17:14:07 - INFO - Train metrics: 0.08725818732104505 | Validation metrics: 0.08785114873840351
10-16 17:14:07 - INFO - Train metrics: 0.014772731283292137 | Validation metrics: 0.012838041080594664
10-16 17:14:07 - INFO - Train metrics: 0.012642252562394865 | Validation metrics: 0.01133115797519985
10-16 17:14:07 - INFO - Train metrics: 0.006194032513930983 | Validation metrics: 0.006271749457363227
10-16 17:14:07 - INFO - Train metrics: 0.004361198887691126 | Validation metrics: 0.004179794604730124
10-16 17:14:07 - INFO - Train metrics: 0.0035579766696529056 | Validation metrics: 0.002832928253808106
10-16 17:14:07 - INFO - Train metrics: 0.002758417570295937 | Validation metrics: 0.002376940750488729
10-16 17:14:07 - INFO - Train metrics: 0.0023976890472368452 | Validation metrics: 0.002198632192713293
10-16 17:14:07 - INFO - Train metrics: 0.0021797855548621307 | Validation metrics: 0.0018213529450198014
10-16 17:14:07 - INFO - Train metrics: 0.001908333388852763 | Validation metrics: 0.001618027281382996
10-16 17:14:07 - INFO - Train metrics: 0.0017698068433746994 | Validation metrics: 0.0015640361352136942
10-16 17:14:07 - INFO - Train metrics: 0.0016177372763605607 | Validation metrics: 0.0013819947730599328
10-16 17:14:07 - INFO - Train metrics: 0.0014788226847697917 | Validation metrics: 0.0012771096786089015
10-16 17:14:07 - INFO - Train metrics: 0.0013776209208598467 | Validation metrics: 0.0012050100232973092
10-16 17:14:07 - INFO - Train metrics: 0.0012723708965577522 | Validation metrics: 0.0011035677308988089
10-16 17:14:07 - INFO - Train metrics: 0.001180949803986785 | Validation metrics: 0.0010317848898903138
10-16 17:14:07 - INFO - Train metrics: 0.0011088856207650331 | Validation metrics: 0.0009732701441724644
10-16 17:14:07 - INFO - Train metrics: 0.0010373910622353804 | Validation metrics: 0.000911393716127019
10-16 17:14:07 - INFO - Train metrics: 0.0009758698419646773 | Validation metrics: 0.0008612624010908408
10-16 17:14:07 - INFO - Train metrics: 0.0009243883591385694 | Validation metrics: 0.0008186591210577524
10-16 17:14:07 - INFO - Train metrics: 0.0008780536535639493 | Validation metrics: 0.0007813527716812913
10-16 17:14:07 - INFO - Train metrics: 0.0008367156797499492 | Validation metrics: 0.0007475953586775847
10-16 17:14:07 - INFO - Train metrics: 0.0008006968379678918 | Validation metrics: 0.0007188091903567465
10-16 17:14:07 - INFO - Train metrics: 0.0007694617914850532 | Validation metrics: 0.000694264061607872
10-16 17:14:07 - INFO - Train metrics: 0.0007423877952104542 | Validation metrics: 0.0006734552355765393
10-16 17:14:07 - INFO - Train metrics: 0.0007186200314220088 | Validation metrics: 0.0006553910136448615
10-16 17:14:07 - INFO - Train metrics: 0.0006980155073841544 | Validation metrics: 0.0006401080043158598
10-16 17:14:07 - INFO - Train metrics: 0.0006800493201238563 | Validation metrics: 0.0006269604988358539
10-16 17:14:07 - INFO - Train metrics: 0.0006643987943787119 | Validation metrics: 0.0006157525472879184
10-16 17:14:07 - INFO - Train metrics: 0.0006504301280821942 | Validation metrics: 0.0006057618655066824
10-16 17:14:07 - INFO - Train metrics: 0.000637476986337145 | Validation metrics: 0.0005963706279719117
10-16 17:14:07 - INFO - Train metrics: 0.000624662876434923 | Validation metrics: 0.0005866550238370293
10-16 17:14:07 - INFO - Train metrics: 0.0006109840260820935 | Validation metrics: 0.0005756043811385152
10-16 17:14:07 - INFO - Train metrics: 0.0005953608785072029 | Validation metrics: 0.0005621059751843638
10-16 17:14:07 - INFO - Train metrics: 0.0005768280936449411 | Validation metrics: 0.0005451675505621942
10-16 17:14:07 - INFO - Train metrics: 0.0005548383808930529 | Validation metrics: 0.0005242279289770081
10-16 17:14:07 - INFO - Train metrics: 0.0005296430529216608 | Validation metrics: 0.0004995457670472637
10-16 17:14:07 - INFO - Train metrics: 0.0005026279740329636 | Validation metrics: 0.00047255352708614536
10-16 17:14:07 - INFO - Train metrics: 0.0004763806951850802 | Validation metrics: 0.00044592941853669333
10-16 17:14:07 - INFO - Train metrics: 0.0004542931422846798 | Validation metrics: 0.0004231917864120932
10-16 17:14:07 - INFO - Train metrics: 0.0004396375663605353 | Validation metrics: 0.0004077585872069867
10-16 17:14:07 - INFO - Train metrics: 0.00043437087082392757 | Validation metrics: 0.00040172051898241417
10-16 17:14:07 - INFO - Train metrics: 0.0004381894049810054 | Validation metrics: 0.00040487633551226346
10-16 17:14:07 - INFO - Train metrics: 0.0004483636926432469 | Validation metrics: 0.0004145573069708365
10-16 17:14:07 - INFO - Train metrics: 0.0004605377795534176 | Validation metrics: 0.0004264386500128441
10-16 17:14:07 - INFO - Train metrics: 0.0004701584572892141 | Validation metrics: 0.00043598471551597345
10-16 17:14:07 - INFO - Train metrics: 0.00047390636965226297 | Validation metrics: 0.00043989907223449063
10-16 17:14:07 - INFO - Train metrics: 0.0004705326842613294 | Validation metrics: 0.0004369586790239231
10-16 17:14:07 - INFO - Train metrics: 0.0004608767757187586 | Validation metrics: 0.00042801723854305845
10-16 17:14:07 - INFO - Train metrics: 0.0004471573065575399 | Validation metrics: 0.0004152822610451793
10-16 17:14:07 - INFO - Train metrics: 0.0004319395329616497 | Validation metrics: 0.0004012696456747374
10-16 17:14:07 - INFO - Train metrics: 0.0004172402004131296 | Validation metrics: 0.00038791711517430916
10-16 17:14:07 - INFO - Train metrics: 0.0004040737772642875 | Validation metrics: 0.00037615611735317443
10-16 17:14:07 - INFO - Train metrics: 0.00039252113014924105 | Validation metrics: 0.0003660025246761212
10-16 17:14:07 - INFO - Train metrics: 0.0003821328100730854 | Validation metrics: 0.00035697119686790184
10-16 17:14:07 - INFO - Train metrics: 0.0003724250152361647 | Validation metrics: 0.00034856507783953214
10-16 17:14:07 - INFO - Train metrics: 0.00036319083556647235 | Validation metrics: 0.00034057357168587094
10-16 17:14:07 - INFO - Train metrics: 0.0003544863816444064 | Validation metrics: 0.000333050251209337
10-16 17:14:07 - INFO - Train metrics: 0.00034638136910638753 | Validation metrics: 0.0003260666527432587
10-16 17:14:07 - INFO - Train metrics: 0.00033880862361287 | Validation metrics: 0.000319561331634024
10-16 17:14:07 - INFO - Train metrics: 0.00033162938109085544 | Validation metrics: 0.0003134027811947909
10-16 17:14:07 - INFO - Train metrics: 0.00032477589052189187 | Validation metrics: 0.0003075269165547621
10-16 17:14:07 - INFO - Train metrics: 0.0003182573017418002 | Validation metrics: 0.0003019435010201326
10-16 17:14:07 - INFO - Train metrics: 0.0003120609272675785 | Validation metrics: 0.000296643210193989
10-16 17:14:07 - INFO - Train metrics: 0.00030613382303746494 | Validation metrics: 0.0002915786636283038
10-16 17:14:07 - INFO - Train metrics: 0.0003004478061128814 | Validation metrics: 0.0002867235489615305
10-16 17:14:07 - INFO - Train metrics: 0.00029499603516070926 | Validation metrics: 0.00028207225018331424
10-16 17:14:07 - INFO - Train metrics: 0.0002897622258948596 | Validation metrics: 0.0002776115006506161
10-16 17:14:08 - INFO - Train metrics: 0.00028472417150915863 | Validation metrics: 0.00027332207404966044
10-16 17:14:08 - INFO - Train metrics: 0.0002798722559594337 | Validation metrics: 0.0002691952527543022
10-16 17:14:08 - INFO - Train metrics: 0.0002751973940364148 | Validation metrics: 0.00026522310685267615
10-16 17:14:08 - INFO - Train metrics: 0.0002706866898255325 | Validation metrics: 0.0002613946196630202
10-16 17:14:08 - INFO - Train metrics: 0.00026633275569507616 | Validation metrics: 0.0002577035285701806
10-16 17:14:08 - INFO - Train metrics: 0.0002621284681360674 | Validation metrics: 0.00025414415553442647
10-16 17:14:08 - INFO - Train metrics: 0.0002580652111017362 | Validation metrics: 0.0002507083879365125
10-16 17:14:08 - INFO - Train metrics: 0.000254140056265322 | Validation metrics: 0.00024739395420777263
10-16 17:14:08 - INFO - Train metrics: 0.00025034411244309113 | Validation metrics: 0.0002441935816243517
10-16 17:14:08 - INFO - Train metrics: 0.00024667496208415453 | Validation metrics: 0.00024110502522260027
10-16 17:14:08 - INFO - Train metrics: 0.00024312857113501995 | Validation metrics: 0.00023812483575764217
10-16 17:14:08 - INFO - Train metrics: 0.00023969987421623316 | Validation metrics: 0.00023524877391849653
10-16 17:14:08 - INFO - Train metrics: 0.00023638762165016183 | Validation metrics: 0.00023247585179180733
10-16 17:14:08 - INFO - Train metrics: 0.00023318712779304436 | Validation metrics: 0.0002298021413893862
10-16 17:14:08 - INFO - Train metrics: 0.00023009817418379752 | Validation metrics: 0.000227227005513733
10-16 17:14:08 - INFO - Train metrics: 0.0002271164587073327 | Validation metrics: 0.0002247471289739784
10-16 17:14:08 - INFO - Train metrics: 0.00022424235063217543 | Validation metrics: 0.00022236294259500662
10-16 17:14:08 - INFO - Train metrics: 0.00022147501331437752 | Validation metrics: 0.00022007321431466163
10-16 17:14:08 - INFO - Train metrics: 0.0002188134007301779 | Validation metrics: 0.00021787796147765045
10-16 17:14:08 - INFO - Train metrics: 0.00021625709021755757 | Validation metrics: 0.0002157763287547336
10-16 17:14:08 - INFO - Train metrics: 0.00021380787851863113 | Validation metrics: 0.00021376921233810448
10-16 17:14:08 - INFO - Train metrics: 0.00021146546529499783 | Validation metrics: 0.00021185746961484918
10-16 17:14:08 - INFO - Train metrics: 0.00020923243987579413 | Validation metrics: 0.00021004253607995885
10-16 17:14:08 - INFO - Train metrics: 0.00020711057256168468 | Validation metrics: 0.0002083264262182638
10-16 17:14:08 - INFO - Train metrics: 0.00020510280273013275 | Validation metrics: 0.00020671141997680558
10-16 17:14:08 - INFO - Train metrics: 0.00020321348133084963 | Validation metrics: 0.00020520108477965074
10-16 17:14:08 - INFO - Train metrics: 0.00020144763851233694 | Validation metrics: 0.00020380024598309337
10-16 17:14:08 - INFO - Train metrics: 0.00019981054214084928 | Validation metrics: 0.00020251363854516637
10-16 17:14:08 - INFO - Train metrics: 0.00019831216895840783 | Validation metrics: 0.00020134952648382897
10-16 17:14:08 - INFO - Train metrics: 0.00019695956548378292 | Validation metrics: 0.00020031402586615936
10-16 17:14:08 - INFO - Train metrics: 0.00019576404570341625 | Validation metrics: 0.0001994180062378879
10-16 17:14:08 - INFO - Train metrics: 0.00019474085648678346 | Validation metrics: 0.00019867391389620612
10-16 17:14:08 - INFO - Train metrics: 0.00019390460788383193 | Validation metrics: 0.0001980958412685949
10-16 17:14:08 - INFO - Train metrics: 0.00019327641381178387 | Validation metrics: 0.00019770190152080936
10-16 17:14:08 - INFO - Train metrics: 0.00019288094027754488 | Validation metrics: 0.00019751458074262536
10-16 17:14:08 - INFO - Train metrics: 0.00019274629581892362 | Validation metrics: 0.0001975594714902473
10-16 17:14:08 - INFO - Train metrics: 0.0001929091000205891 | Validation metrics: 0.00019786948490431626
10-16 17:14:08 - INFO - Train metrics: 0.00019341151609043487 | Validation metrics: 0.00019848334289630968
10-16 17:14:08 - INFO - Train metrics: 0.00019430609482380196 | Validation metrics: 0.00019944892508904402
10-16 17:14:08 - INFO - Train metrics: 0.00019565414792547622 | Validation metrics: 0.00020082274656460592
10-16 17:14:08 - INFO - Train metrics: 0.00019753135735359285 | Validation metrics: 0.00020267466709930263
10-16 17:14:08 - INFO - Train metrics: 0.00020002988578499595 | Validation metrics: 0.00020509070041824622
10-16 17:14:08 - INFO - Train metrics: 0.00020325422246285707 | Validation metrics: 0.00020816815738340443
10-16 17:14:08 - INFO - Train metrics: 0.00020733109027915226 | Validation metrics: 0.0002120264610063962
10-16 17:14:08 - INFO - Train metrics: 0.00021240513669361913 | Validation metrics: 0.0002168013743357733
10-16 17:14:08 - INFO - Train metrics: 0.00021863649360734017 | Validation metrics: 0.00022264595368096248
10-16 17:14:08 - INFO - Train metrics: 0.00022620169971345977 | Validation metrics: 0.0002297286392246006
10-16 17:14:08 - INFO - Train metrics: 0.0002352703104270062 | Validation metrics: 0.00023821362816361767
10-16 17:14:08 - INFO - Train metrics: 0.00024599312447369764 | Validation metrics: 0.0002482478433956552
10-16 17:14:08 - INFO - Train metrics: 0.0002584554883986968 | Validation metrics: 0.0002599180334440498
10-16 17:14:08 - INFO - Train metrics: 0.0002726227152137959 | Validation metrics: 0.00027319689435917284
10-16 17:14:08 - INFO - Train metrics: 0.0002882654773115757 | Validation metrics: 0.0002878726940105829
10-16 17:14:08 - INFO - Train metrics: 0.00030486028383831424 | Validation metrics: 0.00030345068565414596
10-16 17:14:08 - INFO - Train metrics: 0.000321526717882426 | Validation metrics: 0.0003190934142118997
10-16 17:14:08 - INFO - Train metrics: 0.00033699569196374925 | Validation metrics: 0.00033358852229095203
10-16 17:14:08 - INFO - Train metrics: 0.00034970465942780793 | Validation metrics: 0.00034544168062498697
10-16 17:14:08 - INFO - Train metrics: 0.0003580324868799974 | Validation metrics: 0.0003531036917308364
10-16 17:14:08 - INFO - Train metrics: 0.00036067376088323 | Validation metrics: 0.00035532990927958504
10-16 17:14:08 - INFO - Train metrics: 0.00035701441782220897 | Validation metrics: 0.00035154491350186445
10-16 17:14:08 - INFO - Train metrics: 0.00034735165127928845 | Validation metrics: 0.0003420527819532318
10-16 17:14:08 - INFO - Train metrics: 0.0003328743030551709 | Validation metrics: 0.00032801337768276683
10-16 17:14:08 - INFO - Train metrics: 0.00031534414033797617 | Validation metrics: 0.0003111335089385528
10-16 17:14:08 - INFO - Train metrics: 0.0002966750246725911 | Validation metrics: 0.0002932556885155859
10-16 17:14:08 - INFO - Train metrics: 0.0002785147950263288 | Validation metrics: 0.0002759558707534458
10-16 17:14:08 - INFO - Train metrics: 0.0002620418459459448 | Validation metrics: 0.0002603436117013446
10-16 17:14:08 - INFO - Train metrics: 0.00024792235718111046 | Validation metrics: 0.00024702776025867826
10-16 17:14:08 - INFO - Train metrics: 0.00023641143168350753 | Validation metrics: 0.0002362197669248821
10-16 17:14:08 - INFO - Train metrics: 0.00022748184814450463 | Validation metrics: 0.00022786297521441045
10-16 17:14:08 - INFO - Train metrics: 0.0002209824676736886 | Validation metrics: 0.00022178812386383387
10-16 17:14:08 - INFO - Train metrics: 0.00021671872635028864 | Validation metrics: 0.00021779302900393652
10-16 17:14:08 - INFO - Train metrics: 0.0002145225393780178 | Validation metrics: 0.00021570834838032646
10-16 17:14:08 - INFO - Train metrics: 0.0002142844783237909 | Validation metrics: 0.00021542500509913408
10-16 17:14:08 - INFO - Train metrics: 0.00021595909091863397 | Validation metrics: 0.00021689875060786502
10-16 17:14:08 - INFO - Train metrics: 0.00021957483170040532 | Validation metrics: 0.00022015673642851073
10-16 17:14:08 - INFO - Train metrics: 0.00022522427674434277 | Validation metrics: 0.00022528921662608745
10-16 17:14:08 - INFO - Train metrics: 0.000233056085608155 | Validation metrics: 0.0002324395868021788
10-16 17:14:08 - INFO - Train metrics: 0.00024324641484221008 | Validation metrics: 0.0002417778491919785
10-16 17:14:09 - INFO - Train metrics: 0.0002559713402501585 | Validation metrics: 0.00025347396972438677
10-16 17:14:09 - INFO - Train metrics: 0.0002713323037801217 | Validation metrics: 0.0002676267594315413
10-16 17:14:09 - INFO - Train metrics: 0.000289280656519472 | Validation metrics: 0.0002841906756694156
10-16 17:14:09 - INFO - Train metrics: 0.0003094648430882521 | Validation metrics: 0.0003028301180533199
10-16 17:14:09 - INFO - Train metrics: 0.00033108944442113836 | Validation metrics: 0.00032278388207119824
10-16 17:14:09 - INFO - Train metrics: 0.00035278298754949953 | Validation metrics: 0.0003427432525451436
10-16 17:14:09 - INFO - Train metrics: 0.0003725981831003242 | Validation metrics: 0.0003608587533682397
10-16 17:14:09 - INFO - Train metrics: 0.0003882006582025588 | Validation metrics: 0.00037493236654560373
10-16 17:14:09 - INFO - Train metrics: 0.0003972756941155075 | Validation metrics: 0.00038281594217293944
10-16 17:14:09 - INFO - Train metrics: 0.0003980959853581048 | Validation metrics: 0.0003829544493599296
10-16 17:14:09 - INFO - Train metrics: 0.00038999209488580996 | Validation metrics: 0.0003748159015414804
10-16 17:14:09 - INFO - Train metrics: 0.00037358318852904934 | Validation metrics: 0.0003590812378050289
10-16 17:14:09 - INFO - Train metrics: 0.000350699019513296 | Validation metrics: 0.0003375370989405465
10-16 17:14:09 - INFO - Train metrics: 0.0003239749978211263 | Validation metrics: 0.000312673301033845
10-16 17:14:09 - INFO - Train metrics: 0.0002962387593117027 | Validation metrics: 0.00028710571091829077
10-16 17:14:09 - INFO - Train metrics: 0.0002699210986122722 | Validation metrics: 0.0002630356683941426
10-16 17:14:09 - INFO - Train metrics: 0.00024666704409166397 | Validation metrics: 0.00024191144444612843
10-16 17:14:09 - INFO - Train metrics: 0.0002272758773092722 | Validation metrics: 0.00022439685864185895
10-16 17:14:09 - INFO - Train metrics: 0.0002118763173224269 | Validation metrics: 0.0002105485532089428
10-16 17:14:09 - INFO - Train metrics: 0.0002001914796247299 | Validation metrics: 0.00020007121407854926
10-16 17:14:09 - INFO - Train metrics: 0.00019177615351217536 | Validation metrics: 0.00019253388172452986
10-16 17:14:09 - INFO - Train metrics: 0.0001861701399904272 | Validation metrics: 0.00018750625807142845
10-16 17:14:09 - INFO - Train metrics: 0.00018299229970881084 | Validation metrics: 0.00018463991264897314
10-16 17:14:09 - INFO - Train metrics: 0.00018197983937492048 | Validation metrics: 0.00018369910384250825
10-16 17:14:09 - INFO - Train metrics: 0.000182989125348917 | Validation metrics: 0.0001845595741708001
10-16 17:14:09 - INFO - Train metrics: 0.00018600421759415475 | Validation metrics: 0.0001872151969835595
10-16 17:14:09 - INFO - Train metrics: 0.00019112719816067036 | Validation metrics: 0.00019176912345838818
10-16 17:14:09 - INFO - Train metrics: 0.0001985772370908148 | Validation metrics: 0.00019843434762665908
10-16 17:14:09 - INFO - Train metrics: 0.00020868410605804196 | Validation metrics: 0.0002075281047680876
10-16 17:14:09 - INFO - Train metrics: 0.0002218758950551463 | Validation metrics: 0.00021945985167574214
10-16 17:14:09 - INFO - Train metrics: 0.00023864650337917164 | Validation metrics: 0.00023470125222846752
10-16 17:14:09 - INFO - Train metrics: 0.00025949318529127686 | Validation metrics: 0.000253727361048346
10-16 17:14:09 - INFO - Train metrics: 0.00028477633122633676 | Validation metrics: 0.000276879414667921
10-16 17:14:09 - INFO - Train metrics: 0.000314464650810101 | Validation metrics: 0.00030412475109297895
10-16 17:14:09 - INFO - Train metrics: 0.0003477911170038691 | Validation metrics: 0.0003347226632864807
10-16 17:14:09 - INFO - Train metrics: 0.00038283271270099197 | Validation metrics: 0.0003668348804102138
10-16 17:14:09 - INFO - Train metrics: 0.0004162724351744987 | Validation metrics: 0.000397313346514817
10-16 17:14:09 - INFO - Train metrics: 0.00044367487430292413 | Validation metrics: 0.00042199814043036014
10-16 17:14:09 - INFO - Train metrics: 0.0004604463581489469 | Validation metrics: 0.0004366670900984018
10-16 17:14:09 - INFO - Train metrics: 0.00046314447558015807 | Validation metrics: 0.00043827960252376143
10-16 17:14:09 - INFO - Train metrics: 0.0004504776832864697 | Validation metrics: 0.0004258570975666358
10-16 17:14:09 - INFO - Train metrics: 0.0004236750844615117 | Validation metrics: 0.0004007269000996732
10-16 17:14:09 - INFO - Train metrics: 0.000386336288414312 | Validation metrics: 0.0003663048360113412
10-16 17:14:09 - INFO - Train metrics: 0.00034369924112998023 | Validation metrics: 0.0003274037991297629
10-16 17:14:09 - INFO - Train metrics: 0.0003012492788114111 | Validation metrics: 0.0002889760137556798
10-16 17:14:09 - INFO - Train metrics: 0.0002632512650300835 | Validation metrics: 0.00025480634793333443
10-16 17:14:09 - INFO - Train metrics: 0.00023195351846574486 | Validation metrics: 0.00022682607775986798
10-16 17:14:09 - INFO - Train metrics: 0.0002077935820207336 | Validation metrics: 0.00020533519170265832
10-16 17:14:09 - INFO - Train metrics: 0.0001900722575767834 | Validation metrics: 0.00018963407424650146
10-16 17:14:09 - INFO - Train metrics: 0.0001776467180938663 | Validation metrics: 0.0001786530192475766
10-16 17:14:09 - INFO - Train metrics: 0.0001693703446821438 | Validation metrics: 0.0001713423012971003
10-16 17:14:09 - INFO - Train metrics: 0.00016430021787527949 | Validation metrics: 0.00016685025272608706
10-16 17:14:09 - INFO - Train metrics: 0.00016175194811641954 | Validation metrics: 0.0001645643047099425
10-16 17:14:09 - INFO - Train metrics: 0.0001612818323018585 | Validation metrics: 0.0001640935253376621
10-16 17:14:09 - INFO - Train metrics: 0.00016264715152448347 | Validation metrics: 0.0001652294426957719
../_images/narx_neural_network_22_2001.png