VFS Federation Studio 1.1.5 reviewed main assistant benchmark

Rows: 116
Accuracy: 0.9310
Log loss: 0.414864
Top-3 accuracy: 0.9914

class                     precision     recall         F1    support
--------------------------------------------------------------------
greeting                     1.0000     1.0000     1.0000          4
syntax                       0.8000     1.0000     0.8889          4
data                         1.0000     1.0000     1.0000          4
references                   1.0000     0.7500     0.8571          4
context                      1.0000     1.0000     1.0000          4
regex                        1.0000     1.0000     1.0000          4
files                        0.8000     1.0000     0.8889          4
modules                      1.0000     1.0000     1.0000          4
oop                          0.8000     1.0000     0.8889          4
serialization                0.7500     0.7500     0.7500          4
database                     1.0000     1.0000     1.0000          4
testing                      1.0000     0.7500     0.8571          4
errors                       0.7500     0.7500     0.7500          4
debugging                    1.0000     1.0000     1.0000          4
performance                  1.0000     1.0000     1.0000          4
security                     1.0000     0.7500     0.8571          4
concurrency                  1.0000     1.0000     1.0000          4
windows                      1.0000     1.0000     1.0000          4
cli                          0.8000     1.0000     0.8889          4
packaging                    1.0000     1.0000     1.0000          4
web                          1.0000     1.0000     1.0000          4
functions                    1.0000     1.0000     1.0000          4
install                      1.0000     0.7500     0.8571          4
vfs_build                    0.7500     0.7500     0.7500          4
monk_policy                  1.0000     0.7500     0.8571          4
monk_loom                    1.0000     1.0000     1.0000          4
monk_spindle                 1.0000     1.0000     1.0000          4
vfs_perl_runtime             0.8000     1.0000     0.8889          4
vfs_agent                    1.0000     1.0000     1.0000          4
--------------------------------------------------------------------
macro average                0.9397     0.9310     0.9304        116
Weighted F1: 0.9304

Confusion matrix (rows=actual, columns=predicted):
          0     1     2     3     4     5     6     7     8     9    10    11    12    13    14    15    16    17    18    19    20    21    22    23    24    25    26    27    28
   0      4     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0
   1      0     4     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0
   2      0     0     4     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0
   3      0     0     0     3     0     0     0     0     0     0     0     0     1     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0
   4      0     0     0     0     4     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0
   5      0     0     0     0     0     4     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0
   6      0     0     0     0     0     0     4     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0
   7      0     0     0     0     0     0     0     4     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0
   8      0     0     0     0     0     0     0     0     4     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0
   9      0     0     0     0     0     0     0     0     1     3     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0
  10      0     0     0     0     0     0     0     0     0     0     4     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0
  11      0     0     0     0     0     0     0     0     0     0     0     3     0     0     0     0     0     0     1     0     0     0     0     0     0     0     0     0     0
  12      0     1     0     0     0     0     0     0     0     0     0     0     3     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0
  13      0     0     0     0     0     0     0     0     0     0     0     0     0     4     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0
  14      0     0     0     0     0     0     0     0     0     0     0     0     0     0     4     0     0     0     0     0     0     0     0     0     0     0     0     0     0
  15      0     0     0     0     0     0     1     0     0     0     0     0     0     0     0     3     0     0     0     0     0     0     0     0     0     0     0     0     0
  16      0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     4     0     0     0     0     0     0     0     0     0     0     0     0
  17      0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     4     0     0     0     0     0     0     0     0     0     0     0
  18      0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     4     0     0     0     0     0     0     0     0     0     0
  19      0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     4     0     0     0     0     0     0     0     0     0
  20      0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     4     0     0     0     0     0     0     0     0
  21      0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     4     0     0     0     0     0     0     0
  22      0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     3     0     0     0     0     1     0
  23      0     0     0     0     0     0     0     0     0     1     0     0     0     0     0     0     0     0     0     0     0     0     0     3     0     0     0     0     0
  24      0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     1     3     0     0     0     0
  25      0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     4     0     0     0
  26      0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     4     0     0
  27      0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     4     0
  28      0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     4

Class index key:
  0 = greeting
  1 = syntax
  2 = data
  3 = references
  4 = context
  5 = regex
  6 = files
  7 = modules
  8 = oop
  9 = serialization
  10 = database
  11 = testing
  12 = errors
  13 = debugging
  14 = performance
  15 = security
  16 = concurrency
  17 = windows
  18 = cli
  19 = packaging
  20 = web
  21 = functions
  22 = install
  23 = vfs_build
  24 = monk_policy
  25 = monk_loom
  26 = monk_spindle
  27 = vfs_perl_runtime
  28 = vfs_agent
