

.. mlp_summary

.. _ref_to_mlp_summary:


Multilayer Perceptron
=====================

Multi-layer Perceptron (MLP) is a supervised learning algorithm that learns a function 
by training on a dataset. It can have multiple input variables and create multiple output values.

Given a set of features and a target , it can learn a non-linear function approximator.

See the discussion on `Wikipedia <https://en.wikipedia.org/wiki/Multilayer_perceptron>`_
and the `sklearn <https://scikit-learn.org/stable/index.html>`_ implementation at
`Multi-layer Perceptron regressor <https://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPRegressor.html>`_.

Monte Carlo Method
------------------

Many of the calculated parameters required by **RocketIsp** are best calculated by large complex numerical models.
Since including those models in **RocketIsp** is not feasible, correlations of those models' output is used.

**RocketIsp** uses `Multi-layer Perceptron regressors <https://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPRegressor.html>`_
to correlate data generated by complex numerical models.
Each model is treated as a `Black Box <https://en.wikipedia.org/wiki/Black_box>`_ and 
a large random sampling of inputs to the model are used to generate a large sampling of results.
The results are then correlated with a `Multi-layer Perceptron regressor <https://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPRegressor.html>`_.

Such a data-generation approach is known as a `Monte Carlo <https://en.wikipedia.org/wiki/Monte_Carlo_method>`_ method.


Example
-------

In order to demonstrate the method, the large complex numerical model
`Black Box <https://en.wikipedia.org/wiki/Black_box>`_ 
will be replaced by a very simple model, `Monte Carlo <https://en.wikipedia.org/wiki/Monte_Carlo_method>`_
data will be generated from that simple model and that data will be fitted with a 
`Multi-layer Perceptron regressor <https://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPRegressor.html>`_.

For this example
the `Black Box <https://en.wikipedia.org/wiki/Black_box>`_ will be just a single 
equation taken from the `Expanded Liquid Engine Simulation (ELES) <./_static/ELES_Paper_AIAA-85-1385.pdf>`_ computer program.
ELES was created in 1984 by Aerojet Techsystems Company (now `Aerojet Rocketdyne <https://www.rocket.com/>`_)
to model the size, weight and performance 
of liquid propellant stages. An `AIAA paper <./_static/ELES_Paper_AIAA-85-1385.pdf>`_ describes ELES in overview.


On page 13 of the `ELES Technical Information Manual <./_static/ELES_Technical_Information_Manual.pdf>`_
is a rough approximation equation for boundary layer efficiency, :math:`\large{\eta_{bl}}`. 
This example will use that equation to simulate a large complex model.

.. image:: ./_static/etabl_from_eles_tech_manual.jpg

The above :math:`\large{\eta_{bl}}` equation can be implemented in python code as the following function.

.. code-block:: python

    def eles_effbl( Pc=100.0, Fvac=1000.0, eps=20.0):
        """
        Return effBL from ELES boundary layer correlation.
        where: Pc=psia, Fvac=lbf, eps=dimensionless
        """
        logPF   = log(0.01*Pc*Fvac)
        effBL = 0.997 - (log(eps) * 0.01 * (1-0.065*logPF + 0.001*logPF**2))
        return effBL

Create Dataset
~~~~~~~~~~~~~~

One of the considerations when generating data for a 
`Monte Carlo <https://en.wikipedia.org/wiki/Monte_Carlo_method>`_ analysis is the selection
of distribution type for each input parameter. In general a
`continuous uniform distribution <https://en.wikipedia.org/wiki/Continuous_uniform_distribution>`_
is good for this application, however, when an input can cover 
`orders of magnitude <https://en.wikipedia.org/wiki/Order_of_magnitude>`_ ,
it is sometimes better to use a uniform distribution of the 
`logarithm <https://en.wikipedia.org/wiki/Logarithm>`_ of that input. For this example, distributions of the 
`logarithm <https://en.wikipedia.org/wiki/Logarithm>`_ of chamber pressure, vacuum thrust and area ratio were used,

A small python code (expand section below) 
was used to make a `pandas <https://pandas.pydata.org/docs/getting_started/overview.html>`_
dataframe of Monte Carlo data and save it to a 
`CSV file <https://en.wikipedia.org/wiki/Comma-separated_values>`_.
This is the data that will be fitted with a 
`Multi-layer Perceptron regressor <https://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPRegressor.html>`_.

.. raw:: html

   <details style="font-size: 120%; margin: 10px 10px 10px 10px; cursor: pointer; font-style: italic; color:green">
   <summary>
    Click HERE to view python code
   </summary>

.. literalinclude:: ./_static/example_scripts/mlp_example_make_csv.py
    :language: python

.. raw:: html

   </details>

A display of the `pandas <https://pandas.pydata.org/docs/getting_started/overview.html>`_ dataframe description is
shown below. Notice that **timeOfDay** is included to act as an uncorrelated input.

.. code-block:: python

                  effBL     log10(Pc)   log10(Fvac)    log10(eps)     timeOfDay
    count  10000.000000  10000.000000  10000.000000  10000.000000  10000.000000
    mean       0.981543      2.740944      3.066303      1.296131     12.077017
    std        0.008608      0.422945      1.379687      0.580750      6.920006
    min        0.951005      2.000074      0.699824      0.301047      0.003217
    25%        0.975976      2.380903      1.857840      0.789792      6.126944
    50%        0.982975      2.748261      3.061789      1.297682     12.051867
    75%        0.988453      3.103367      4.261623      1.803856     18.014434
    max        0.995327      3.477062      5.476351      2.300927     23.999201

Select Independent Inputs
~~~~~~~~~~~~~~~~~~~~~~~~~

While it is possible to simply use all of the complex model's inputs as inputs to the 
`Multi-layer Perceptron regressor <https://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPRegressor.html>`_,
it is probably better to select only those inputs that make a difference in the output, :math:`\large{\eta_{bl}}`.

One way to learn, not only which inputs matter, but their order of importance, is the following:

One by one, fit each individual independent parameter to the dataset with a
`Multi-layer Perceptron regressor <https://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPRegressor.html>`_.
Select the input with the best correlation coefficient as the most important input.

Next combine the most important input with the remaining inputs, one by one, to find the second most important
input. Continue the process until all possible inputs are in order of importance.

The chart below shows the results of this process on the example dataset.
It indicates that AreaRatio and Thrust are both very important. Pc adds some improvement and timeOfDay adds nothing.

As a result, it seems reasonable to use **AreaRatio, Thrust and Pc** as inputs to the final
`Multi-layer Perceptron regressor <https://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPRegressor.html>`_.
(knowing what is in the `Black Box <https://en.wikipedia.org/wiki/Black_box>`_ we should have expected those
3 inputs, but we may not have known their order of importance)

.. image:: ./_static/mlp_example_plot_dep.png
    :width: 49%




Fit MLP to Dataset
~~~~~~~~~~~~~~~~~~

A small python script was used to fit the monte carlo data to a
`Multi-layer Perceptron regressor <https://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPRegressor.html>`_


.. raw:: html

   <details style="font-size: 120%; margin: 10px 10px 10px 10px; cursor: pointer; font-style: italic; color:green">
   <summary>
    Click HERE to view python code
   </summary>

.. literalinclude:: ./_static/example_scripts/mlp_example_fit.py

.. raw:: html

   </details>

The script outputs a summary of the scaled inputs and output.
Note that all of the inputs and the output are scaled to between 0 and 1.
Also note that the correlation coefficient is excellent... it is greater than 0.999 (will vary slightly with each run).
The script saves the MLP regressor to a file named "mlp_example.joblib" that can be reloaded in order to make use
of the correlation.::

    fitting eff   with indepL = ['Pc', 'Fvac', 'eps']
                     Pc          Fvac           eps
    count  10000.000000  10000.000000  10000.000000
    mean       0.493963      0.493261      0.498065
    std        0.281963      0.275937      0.290375
    min        0.000049      0.019965      0.000523
    25%        0.253935      0.251568      0.244896
    50%        0.498841      0.492358      0.498841
    75%        0.735578      0.732325      0.751928
    max        0.984708      0.975270      1.000463
    --------------------------------------------
    count    10000.000000
    mean         0.630870
    std          0.172157
    min          0.020108
    25%          0.519511
    50%          0.659495
    75%          0.769058
    max          0.906547
    Name: eff, dtype: float64
    --------------------------------------------
    correlation coefficient: 0.9998603014551914

Use The Regressor
~~~~~~~~~~~~~~~~~


Our black box was very well behaved and the MLP fit was excellent.
The charts below demonstrate an excellent agreement between the original black box data
and the MLP prediction.

In general, if the black box being examined is as well behaved as this example, similarly good
results will likely be realized.
Complex numerical models may have much more numerical noise due to things like iteration termination criteria, 
local optima, calculation uncertainties, numerically induced oscillations, internal discontinuities, etc.
In those cases, the MLP fit will still likely reflect the character of the underlying data,
however, the correlation coefficient of the fit will likely be below the value of this example

The following charts compare the MLP regressor values to the original values of
the `Black Box <https://en.wikipedia.org/wiki/Black_box>`_.



.. raw:: html

    <table width="100%">
    <tr>
    <th style="text-align:center;"> MLP Fit at Pc=100 psia </th>
    <th style="text-align:center;"> MLP Fit at Pc=1000 psia </th>
    </tr>
    <tr>
    <td width="50%">
    <a class="reference internal image-reference" href="./_static/mlp_example_Pc=100.png">
    <img src="./_static/mlp_example_Pc=100.png">
    </a>
    </td>
    <td>
    <a class="reference internal image-reference" href="./_static/mlp_example_Pc=1000.png">
    <img src="./_static/mlp_example_Pc=1000.png">
    </a>
    </td>
    </tr>
    <tr>
    <td colspan="2" style="text-align:center;">
    <p><cite>Click image to see full size</cite></p>
    </td>
    </tr>
    </table>
    

The following python script reads the MLP regressor from its saved file and uses it to predict the 
original values of the `Black Box <https://en.wikipedia.org/wiki/Black_box>`_.

.. literalinclude:: ./_static/example_scripts/mlp_example_predict.py

