LBL logo

Previous topic

Examples

Next topic

Copyright and License

Development

This module contains the class Tester that runs the unit tests of the Buildings library, and the class Validator that validates the html code of the info section of the .mo files.

class buildingspy.development.regressiontest.Tester(**kwargs)

Class that runs all regression tests using Dymola.

Initiate with the following optional arguments:

Parameters:
  • checkHtml – bool (default=True). Specify whether to load tidylib and perform validation of html documentation
  • executable – {‘dymola’, ‘omc’}. Default is ‘dymola’, specifies the executable to use for running the regression test with run().
  • cleanup – bool (default=True). Specify whether to delete temporary directories.

This class can be used to run all regression tests. It searches the directory CURRENT_DIRECTORY\Resources\Scripts\Dymola for all *.mos files that contain the string simulate, where CURRENT_DIRECTORY is the name of the directory in which the Python script is started, as returned by the function getLibraryName(). All these files will be executed as part of the regression tests. Any variables or parameters that are plotted by these *.mos files will be compared to previous results that are stored in CURRENT_DIRECTORY\Resources\ReferenceResults\Dymola. If no reference results exist, then they will be created. Otherwise, the accuracy of the new results is compared to the reference results. If they differ by more than a prescibed tolerance, a plot such as the one below is shown.

_images/unitTestPlot.png

Plot that compares the new results (solid line) of the regression test with the old results (dotted line). The blue line indicates the time where the largest error occurs.

In this plot, the vertical line indicates the time where the biggest error occurs. The user is then asked to accept or reject the new results.

To run the regression tests, type

>>> import os
>>> import buildingspy.development.regressiontest as r
>>> rt = r.Tester()
>>> myMoLib = os.path.join("buildingspy", "tests", "MyModelicaLibrary")
>>> rt.setLibraryRoot(myMoLib)
>>> rt.run() 
Using  1  of ... processors to run unit tests.
Number of models   :  1
          blocks   :  0
          functions:  0
Generated  1  regression tests.

Script that runs unit tests had 0 warnings and 0 errors.

See 'unitTests.log' for details.
Unit tests completed successfully.

Execution time = ...

To run regression tests only for a single package, call setSinglePackage() prior to run().

areResultsEqual(tOld, yOld, tNew, yNew, varNam, filNam)

Return True if the data series are equal within a tolerance.

Parameters:
  • tOld – List of old time values.
  • yOld – Old simulation results.
  • tNew – Time stamps of new results.
  • yNew – New simulation results.
  • varNam – Variable name, used for reporting.
  • filNam – File name, used for reporting.
Returns:

A list with False if the results are not equal, and the time of the maximum error, and a warning message or None. In case of errors, the time of the maximum error may by None.

batchMode(batchMode)

Set the batch mode flag.

Parameters:batchMode – Set to True to run without interactive prompts and without plot windows.

By default, the regression tests require the user to respond if results differ from previous simulations. This method can be used to run the script in batch mode, suppressing all prompts that require the user to enter a response. If run in batch mode, no new results will be stored. To run the regression tests in batch mode, enter

>>> import os
>>> import buildingspy.development.regressiontest as r
>>> r = r.Tester()
>>> r.batchMode(True)
>>> r.run() 
checkPythonModuleAvailability()

Check whether all required python modules are installed.

If some modules are missing, then an ImportError is raised.

deleteTemporaryDirectories(delete)

Flag, if set to False, then the temporary directories will not be deleted after the regression tests are run.

Parameters:delete – Flag, set to False to avoid the temporary directories to be deleted.

Unless this method is called prior to running the regression tests with delete=False, all temporary directories will be deleted after the regression tests.

getLibraryName()

Return the name of the library that will be run by this regression test.

Returns:The name of the library that will be run by this regression test.
getModelicaCommand()

Return the name of the modelica executable.

Returns:The name of the modelica executable.
isExecutable(program)

Return True if the program is an executable

isValidLibrary()

Returns true if the regression tester points to a valid library that implements the scripts for the regression tests.

“return: True if the library implements regression tests, False otherwise.

printNumberOfClasses()

Print the number of models, blocks and functions to the standard output stream

run()

Run all regression tests and checks the results.

Returns:0 if no errros occurred during the regression tests, otherwise a non-zero value.

This method

  • creates temporary directories for each processors,
  • copies the directory CURRENT_DIRECTORY into these temporary directories,
  • creates run scripts that run all regression tests,
  • runs these regression tests,
  • collects the dymola log files from each process,
  • writes the combined log file dymola.log to the current directory,
  • compares the results of the new simulations with reference results that are stored in Resources/ReferenceResults,
  • writes the message Regression tests completed successfully. if no error occured,
  • returns 0 if no errors occurred, or non-zero otherwise.
setDataDictionary()

Build the data structures that are needed to parse the output files.

setLibraryRoot(rootDir)

Set the root directory of the library.

Parameters:rootDir – The top-most directory of the library.

The root directory is the directory that contains the Resources folder and the top-level package.mo file.

Usage: Type
>>> import os
>>> import buildingspy.development.regressiontest as r
>>> rt = r.Tester()
>>> myMoLib = os.path.join("buildingspy", "tests", "MyModelicaLibrary")
>>> rt.setLibraryRoot(myMoLib)
setNumberOfThreads(number)

Set the number of parallel threads that are used to run the regression tests.

Parameters:number – The number of parallel threads that are used to run the regression tests.

By default, the number of parallel threads are set to be equal to the number of processors of the computer.

setSinglePackage(packageName)

Set the name of a single Modelica package to be tested.

Parameters:packageName – The name of the package to be tested.

Calling this method will cause the regression tests to run only for the examples in the package packageName, and in all its sub-packages. For example, if packageName = Annex60.Fluid.Sensors, then a test of the Annex60 library will run all examples in Annex60.Fluid.Sensors.*.

testOM(cmpl=True, simulate=False, packages=['Examples'], number=-1)

Test the library compliance with OpenModelica.

This is the high-level method to test a complete library, even if there are no specific .mos files in the library for regression testing.

This method sets self._nPro to 1 as it only works on a single core. It also executes self.setTemporaryDirectories()

Parameters:
  • cpml – Set to True for the model to be compiled.
  • simulate – Set to True to cause the model to be simulated (from 0 to 1s).
  • packages – Set to a list whose elements are the packages that contain the test models of the library
  • number – Number of models to test. Set to -1 to test all models.

Usage:

  1. In a python console or script, cd to the root folder of the library

    >>> t = Tester() 
    >>> t.testOpenModelica(...) 
    
test_JModelica(cmpl=True, load=False, simulate=False, packages=['Examples'], number=-1)

Test the library compliance with JModelica.org.

This is the high-level method to test a complete library, even if there are no specific .mos files in the library for regression testing.

This method sets self._nPro to 1 as it only works on a single core. It also executes self.setTemporaryDirectories()

Parameters:
  • cpml – Set to True for the model to be compiled.
  • load – Set to True to load the model from the FMU.
  • simulate – Set to True to cause the model to be simulated.
  • packages – Set to an array whose elements are the packages that contain the test models of the library.
  • number – Number of models to test. Set to -1 to test all models.

Usage:

  1. Open a JModelica environment (ipython or pylab with JModelica environment variables set).

  2. cd to the root folder of the library

  3. type tye following commands:

    >>> t = Tester() 
    >>> t.testJmodelica(...) 
    
useExistingResults(dirs)

This function allows to use existing results, as opposed to running a simulation.

Parameters:dirs – A non-empty list of directories that contain existing results.

This method can be used for testing and debugging. If called, then no simulation is run. If the directories ['/tmp/tmp-Buildings-0-zABC44', '/tmp/tmp-Buildings-0-zQNS41'] contain previous results, then this method can be used as

>>> import buildingspy.development.regressiontest as r
>>> l=['/tmp/tmp-Buildings-0-zABC44', '/tmp/tmp-Buildings-0-zQNS41']
>>> rt = r.Tester()
>>> rt.useExistingResults(l)
>>> rt.run()