2  Multi-dimensional Functions

This notebook illustrates how high-dimensional functions can be analyzed.

2.1 Example: Spot and the 3-dim Sphere Function

import numpy as np
from spotPython.fun.objectivefunctions import analytical
from spotPython.spot import spot

2.1.1 The Objective Function: 3-dim Sphere

  • The spotPython package provides several classes of objective functions.

  • We will use an analytical objective function, i.e., a function that can be described by a (closed) formula: \[f(x) = \sum_i^n x_i^2 \]

  • Here we will use \(n=3\).

fun = analytical().fun_sphere
  • The size of the lower bound vector determines the problem dimension.

  • Here we will use -1.0 * np.ones(3), i.e., a three-dim function.

  • We will use three different theta values (one for each dimension), i.e., we set

    surrogate_control={"n_theta": 3}.

TensorBoard

Similar to the one-dimensional case, which was introduced in Section Section 1.7, we can use TensorBoard to monitor the progress of the optimization. We will use the same code, only the prefix is different:

from spotPython.utils.file import get_experiment_name
from spotPython.utils.init import fun_control_init
from spotPython.utils.file import get_spot_tensorboard_path

PREFIX = "02"
experiment_name = get_experiment_name(prefix=PREFIX)
print(experiment_name)

fun_control = fun_control_init(
    spot_tensorboard_path=get_spot_tensorboard_path(experiment_name))
02_bartz08-2_2023-07-10_00-19-26
spot_3 = spot.Spot(fun=fun,
                   lower = -1.0*np.ones(3),
                   upper = np.ones(3),
                   var_name=["Pressure", "Temp", "Lambda"],
                   show_progress=True,
                   surrogate_control={"n_theta": 3},
                   fun_control=fun_control,)

spot_3.run()
spotPython tuning: 0.03443367156190887 [#######---] 73.33% 
spotPython tuning: 0.031348911082058686 [########--] 80.00% 
spotPython tuning: 0.0009629115535977041 [#########-] 86.67% 
spotPython tuning: 8.600065786394651e-05 [#########-] 93.33% 
spotPython tuning: 5.9908343748136085e-05 [##########] 100.00% Done...
<spotPython.spot.spot.Spot at 0x1085d41f0>

Now we can start TensorBoard in the background with the following command:

tensorboard --logdir="./runs"

We can access the TensorBoard web server with the following URL:

http://localhost:6006/

2.1.2 Results

spot_3.print_results()
min y: 5.9908343748136085e-05
Pressure: 0.005157864627379999
Temp: 0.00195710957248863
Lambda: 0.005429042121316765
[['Pressure', 0.005157864627379999],
 ['Temp', 0.00195710957248863],
 ['Lambda', 0.005429042121316765]]
spot_3.plot_progress()

2.1.3 A Contour Plot

  • We can select two dimensions, say \(i=0\) and \(j=1\), and generate a contour plot as follows.
    • Note: We have specified identical min_z and max_z values to generate comparable plots!
spot_3.plot_contour(i=0, j=1, min_z=0, max_z=2.25)

  • In a similar manner, we can plot dimension \(i=0\) and \(j=2\):
spot_3.plot_contour(i=0, j=2, min_z=0, max_z=2.25)

  • The final combination is \(i=1\) and \(j=2\):
spot_3.plot_contour(i=1, j=2, min_z=0, max_z=2.25)

  • The three plots look very similar, because the fun_sphere is symmetric.
  • This can also be seen from the variable importance:
spot_3.print_importance()
Pressure:  100.0
Temp:  99.78247670817808
Lambda:  94.72233826625329
[['Pressure', 100.0],
 ['Temp', 99.78247670817808],
 ['Lambda', 94.72233826625329]]

2.1.4 TensorBoard

TensorBoard visualization of the spotPython process. Objective function values plotted against wall time.

The second TensorBoard visualization shows the input values, i.e., \(x_0, \ldots, x_2\), plotted against the wall time. TensorBoard visualization of the spotPython process.

The third TensorBoard plot illustrates how spotPython can be used as a microscope for the internal mechanisms of the surrogate-based optimization process. Here, one important parameter, the learning rate \(\theta\) of the Kriging surrogate is plotted against the number of optimization steps.

TensorBoard visualization of the spotPython surrogate model.

2.2 Conclusion

Based on this quick analysis, we can conclude that all three dimensions are equally important (as expected, because the analytical function is known).

2.3 Exercises

  • Important:
    • Results from these exercises should be added to this document, i.e., you should submit an updated version of this notebook.
    • Please combine your results using this notebook.
    • Only one notebook from each group!
    • Presentation is based on this notebook. No addtional slides are required!
    • spotPython version 0.16.11 (or greater) is required

2.3.1 The Three Dimensional fun_cubed

  • The input dimension is 3. The search range is \(-1 \leq x \leq 1\) for all dimensions.
  • Generate contour plots
  • Calculate the variable importance.
  • Discuss the variable importance:
    • Are all variables equally important?
    • If not:
      • Which is the most important variable?
      • Which is the least important variable?

2.3.2 The Ten Dimensional fun_wing_wt

  • The input dimension is 10. The search range is \(0 \leq x \leq 1\) for all dimensions.
  • Calculate the variable importance.
  • Discuss the variable importance:
    • Are all variables equally important?
    • If not:
      • Which is the most important variable?
      • Which is the least important variable?
    • Generate contour plots for the three most important variables. Do they confirm your selection?

2.3.3 The Three Dimensional fun_runge

  • The input dimension is 3. The search range is \(-5 \leq x \leq 5\) for all dimensions.
  • Generate contour plots
  • Calculate the variable importance.
  • Discuss the variable importance:
    • Are all variables equally important?
    • If not:
      • Which is the most important variable?
      • Which is the least important variable?

2.3.4 The Three Dimensional fun_linear

  • The input dimension is 3. The search range is \(-5 \leq x \leq 5\) for all dimensions.
  • Generate contour plots
  • Calculate the variable importance.
  • Discuss the variable importance:
    • Are all variables equally important?
    • If not:
      • Which is the most important variable?
      • Which is the least important variable?