Skip to main content
`Ctrl`+`K`

# Tutorial 00: SConstruct

## Prerequisites

1. Computational Practices - Build System

2. Software Carpentry: SCons Novice - https://carpentries-incubator.github.io/scons-novice/ [1, 9]

3. Software Carpentry: Python - https://swcarpentry.github.io/python-novice-inflammation/ [1, 8]

## References

- Python style guide: PEP-8 [41]

## Environment

SCons and WAVES can be installed in a Conda environment with the Conda package manager. See the Conda installation and Conda environment management documentation for more details about using Conda.

Note

The SALib and numpy versions may not need to be this strict for most tutorials. However, Tutorial: Sensitivity Study uncovered some undocumented SALib version sensitivity to numpy surrounding the numpy v2 rollout.

1. Create the tutorials environment if it doesn’t exist

   Linux/MacOS

       $ conda create --name waves-tutorial-env --channel conda-forge waves 'scons>=4.6' matplotlib pandas pyyaml xarray seaborn 'numpy>=2' 'salib>=1.5.1' pytest

   Windows

       PS > conda create --name waves-tutorial-env --channel conda-forge waves scons matplotlib pandas pyyaml xarray seaborn numpy salib pytest

2. Activate the environment

   Linux/MacOS

       $ conda activate waves-tutorial-env

   Windows

       PS > conda activate waves-tutorial-env

Some tutorials require additional third-party software that is not available for the Conda package manager. This software must be installed separately and either made available to SConstruct by modifying your system’s `PATH` or by modifying the SConstruct search paths provided to the `waves.scons_extensions.add_program()` method.

Warning

STOP! Before continuing, check that the documentation version matches your installed package version.

1. You can find the documentation version in the upper-left corner of the webpage.

2. You can find the installed WAVES version with `waves --version`.

If they don’t match, you can launch identically matched documentation with the WAVES Command-Line Utility docs subcommand as `waves docs`.

## Description

WAVES is a suite of build wrappers and command-line utilities for the build system SCons. Build systems help automate the execution of complex computing workflows by constructing component task execution order. In the WAVES tutorials there are two components to the SCons project configuration: project configuration and simulation workflow configurations. This tutorial introduces the SCons project configuration file for the WAVES tutorials and ModSim Templates.

The command-line utilities provided by WAVES are those utilities required to implement engineering workflows in traditional software build systems. For most engineering simulation workflows, software build systems will work out-of-the-box. However, it is difficult to implement engineering parameter studies in software build systems, which are designed to produce a single program, not many nearly identical configurations of the same program. The WAVES parameter generator command-line interface(s) is designed to work with most build systems, but were originally developed with the requirements of CMake in mind.

For production-engineering analysis, WAVES focuses on extending the build system SCons because SCons configuration files use Python as a fully featured scripting language. This choice is primarily driven by the familiarity of the engineering community with Python as a programming language, but also because the parameter generation utility can be integrated more closely with the build system via a Python API, Parameter Generators.

This tutorial will build out the project configuration file, named `SConstruct` by default. First a code snippet will be introduced followed by a description of what that snippet does. After sufficient code is added to the file, it can be executed with the `scons` command. Later tutorials will continue to expand the SConstruct file and add workflow configuration files.

## SCons Project Configuration

3. Create and change to a new project root directory to house the tutorial files if you have not already done so. For example

Linux/MacOS

    $ mkdir -p ~/waves-tutorials
    $ cd ~/waves-tutorials
    $ pwd
    /home/roppenheimer/waves-tutorials

Windows

    PS > New-Item $HOME\waves-tutorials -ItemType "Directory"
    PS > Set-Location $HOME\waves-tutorials
    PS > Get-Location

    Path
    ----
    C:\Users\roppenheimer\waves-tutorials

4. Create a new file named `SConstruct` in the `waves-tutorials` directory and add the contents listed below. Note that the filename is case sensitive and does not use a file extension.

waves-tutorials/SConstruct

    1#! /usr/bin/env python
    2
    3import os
    4import pathlib
    5
    6import waves

By convention, the SCons root project file is named `SConstruct`. Because this is a Python file, we can import Python libraries to help define project settings. The shebang in the first line is included to help text editors identify the file as a Python file for syntax highlighting. Using the PEP-8 formatting, the Python built-in imports are listed in the first block and third-party imports are listed in the second block, including the WAVES package.

5. Add the content below to the `SConstruct` file to add the project’s command-line build options to the project configuration.

Note

The line numbers may jump between code snippets. The skipped lines are not important to the tutorial content as you create the tutorial files. They are a side effect of methods used to robustly create these code snippets from the original source code.

waves-tutorials/SConstruct

     9# Accept command line options with fall back default values
    10AddOption(
    11    "--build-dir",
    12    dest="variant_dir_base",
    13    default="build",
    14    nargs=1,
    15    type="string",
    16    action="store",
    17    metavar="DIR",
    18    help="SCons build (variant) root directory. Relative or absolute path. (default: '%default')",
    19)
    20AddOption(
    21    "--unconditional-build",
    22    dest="unconditional_build",
    23    default=False,
    24    action="store_true",
    25    help="Boolean flag to force building of conditionally ignored targets. (default: '%default')",
    26)
    27AddOption(
    28    "--print-build-failures",
    29    dest="print_build_failures",
    30    default=False,
    31    action="store_true",
    32    help="Print task *.stdout target file(s) on build failures. (default: '%default')",
    33)
    34# Python optparse appends to the default list instead of overriding. Must implement default/override ourselves.
    35default_abaqus_commands = [
    36    "/apps/abaqus/Commands/abq2024",
    37    "/usr/projects/ea/abaqus/Commands/abq2024",
    38    "abq2024",
    39    "abaqus",
    40]
    41AddOption(
    42    "--abaqus-command",
    43    dest="abaqus_command",
    44    nargs=1,
    45    type="string",
    46    action="append",
    47    metavar="COMMAND",
    48    help=f"Override for the Abaqus command. Repeat to specify more than one (default: {default_abaqus_commands})",
    49)

The SCons command-line build options are specific to the project configuration that you are currently creating. SCons projects may add or remove command-line options to aid in build behavior control. The most relevant option to modsim projects will be `--build-dir` and `--abaqus-command`, which allow project developers to change the build directory location and Abaqus executable path from the command-line without modifying the `SConstruct` file source code.

For example, when there are workflow tasks defined in the the `SConstruct` file, a call to `scons` would create the default build directory named `build` and a call to `scons --build-dir=non_default_build` would create a build directory named `non_default_build`. The current SConstruct file will not produce these directories yet because there are no workflow tasks defined. The demonstration below shows how the build directory option will behave when workflow tasks are defined.

Warning

The equals sign separator is important to the internal implementation of options defined by `AddOption` in SConstruct files. When whitespace is used to delimit the option from the argument, the argument handling is undefined and may result in unexpected behavior. Interested readers can read more in the SCons documentation: https://scons.org/doc/production/HTML/scons-user.html#app-functions

Linux/MacOS

    $ scons
    $ ls
    build SConstruct
    $ scons --build-dir=non_default_build
    $ ls
    build non_default_build SConstruct

Windows

    PS > scons
    PS > Get-ChildItem

       Directory: C:\Users\roppenheimer\waves-tutorials

    Mode                 LastWriteTime         Length Name
    ----                 -------------         ------ ----
    -a---            6/9/2023  4:32 PM          23004 build
    -a---            6/9/2023  4:32 PM          23004 SConstruct

    PS > scons --build-dir=non_default_build
    PS > Get-ChildItem

       Directory: C:\Users\roppenheimer\waves-tutorials

    Mode                 LastWriteTime         Length Name
    ----                 -------------         ------ ----
    -a---            6/9/2023  4:32 PM          23004 build
    -a---            6/9/2023  4:32 PM          23004 non_default_build
    -a---            6/9/2023  4:32 PM          23004 SConstruct

The `--abaqus-command` option will be useful if the tutorial files struggle to find your Abaqus installation. You can also edit the `default_abaqus_commands` list to include your Abaqus installation to avoid having to provide it on the command line every time you call `scons`.

Linux/MacOS

    $ scons --abaqus-command=/path/to/installed/abaqus

Windows

    PS > scons --abaqus-command=C:\Path\to\installed\abaqus

The `--unconditional-build` option is mostly useful for Regression Testing and continuous integration. It is used in the tutorial workflows to force a workflow to execute, even if the required programs are not found. This is useful for system testing software availability in the construction environment itself.

By default, WAVES builders redirect task STDOUT and STDERR to a `*.stdout` file to avoid cluttering the task execution reported by SCons. The `--print-build-failures` will print the associated `*.stdout` file of all failed tasks at the end of SCons execution.

At the end of this tutorial, you will see how to explore the project specific command-line options help and usage.

6. Add the content below to the `SConstruct` file to initialize the `waves.scons_extensions.WAVESEnvironment` SCons construction environment.

waves-tutorials/SConstruct

    53# Inherit user's full environment and set project variables
    54env = waves.scons_extensions.WAVESEnvironment(
    55    ENV=os.environ.copy(),
    56    variant_dir_base=pathlib.Path(GetOption("variant_dir_base")),
    57    unconditional_build=GetOption("unconditional_build"),
    58    print_build_failures=GetOption("print_build_failures"),
    59    abaqus_commands=GetOption("abaqus_command"),
    60)
    61
    62# Conditionally print failed task *.stdout files
    63env.PrintBuildFailures(print_stdout=env["print_build_failures"])

Most build systems inherit the user’s active shell environment at build configuration time, referred to as the “external” environment in the SCons documentation. This means that (most) of the environment must be identical for all build tasks in the project. SCons differs from most build systems by managing the construction environment for each task separately from the external environment. SCons projects do not inherit the user’s shell environment at build configuration by default. Instead, projects define one or more construction environments that is used to define per-task environment configuration.

While this is a powerful feature for large, complex projects, most modsim projects will benefit from maintaining a single construction environment inherited from the active shell environment at build configuration time. In addition to copying the active external environment, the above code adds the project command-line build options to the construction environment for reuse throughout the project configuration files, SConstruct and SConscript, for build control.

7. Add the content below to the `SConstruct` file to add the third-party software dependency checks to the project configuration.

waves-tutorials/SConstruct

    67# Find required programs for conditional target ignoring and absolute path for use in target actions
    68env["ABAQUS_PROGRAM"] = env.AddProgram(
    69    env["abaqus_commands"] if env["abaqus_commands"] is not None else default_abaqus_commands
    70)

These checks are not strictly required for an SCons WAVES modsim project; however, they provide valuable build control options for developers. Most of the WAVES tutorials compute environment dependencies are Python packages managed with Conda as described in the Environment section of this tutorial. Many modsim repositories will also depend on proprietary or commercial software that is not included in a package manager such as Conda. Instead, the project configuration can check the construction environment for software installation(s) and provide an environment variable to conditionally skip any tasks that depend on missing software.

In WAVES and WAVES tutorials, this approach is primarily used to allow developers to perform development work on local computers without cluttering their test builds with tasks that cannot succeed on their local computer. Tutorial 01: Geometry will introduce the use of these variables for build control.

The SCons native solution for finding a program is the CheckProg configuration method. The `waves.scons_extensions.add_program()` method wraps CheckProg like behavior to search for a list of possible program names. This is most useful when multiple versions of a program can be used to build the project and the various servers where the project is built may have different versions available. The project will build with the first available program name without changes to the project configuration. If a program is found, it will also be added to the SCons environment `PATH`.

The `env.AddProgram` was added to the construction environment with `waves.scons_extensions.WAVESEnvironment` for convenience. Anywhere the construction environment is available, we can use `env.AddProgram`. It is also possible to use the `waves.scons_extensions.add_program()` function directly. Most projects will search for more than one third-party software, so the shortened `env.AddProgram` function call is also less to type.

Note

The Abaqus program naming convention used here is specific to the naming convention used on the WAVES continuous integration server. Users may need to update the abaqus program name to match their system installation.

8. Add the content below to the `SConstruct` file to add the project meta data to the construction environment.

waves-tutorials/SConstruct

    74# Set project internal variables and variable substitution dictionaries
    75project_name = "WAVES-TUTORIAL"
    76version = "0.1.0"
    77project_dir = pathlib.Path(Dir(".").abspath)
    78project_variables = {
    79    "project_name": project_name,
    80    "project_dir": project_dir,
    81    "version": version,
    82}
    83for key, value in project_variables.items():
    84    env[key] = value

The WAVES tutorials make use of the SCons hierarchical build feature to separate simulation output in the build directory. This is valuable for modsim repositories that include a suite of simulations. To avoid hardcoded duplication of project meta data, the project meta data variables are added to the construction environment, which will be passed around to all SCons configuration files. The implementation that passes the construction environment around is introduced in Tutorial 01: Geometry.

9. Add the content below to the `SConstruct` file to add a placeholder for WAVES builders to the project configuration.

waves-tutorials/SConstruct

    88# Add builders and pseudo-builders
    89env.Append(BUILDERS={})

Although it is possible to re-create the WAVES tutorials entirely in native SCons code, the builder extensions provided by WAVES reduce the requisite background knowledge to begin creating modsim repositories. While the `waves.scons_extensions.WAVESEnvironment` construction environment provides some useful default builders, it is possible and often valuable to add project custom-tailored builders, as well. The construction environment `BUILDERS` variable must be updated to include these custom SCons builders and make them available to the simulation configuration starting in Tutorial 04: Simulation.

The WAVES SCons Extensions API describes the available builder factories and their usage. As WAVES matures, more software will be supported with build wrappers. Prior to a WAVES builder, modsim developers can create their own SCons custom builders or start from a WAVES template builder factory in Tutorial: Writing Builders.

10. Add the content below to the `SConstruct` file to create a placeholder call to the hierarchical simulation configuration files.

waves-tutorials/SConstruct

    93# Add simulation targets
    94workflow_configurations = []
    95for workflow in workflow_configurations:
    96    build_dir = env["variant_dir_base"] / workflow
    97    SConscript(workflow, variant_dir=build_dir, exports={"env": env}, duplicate=False)

The for loop in this code-snippet is the method for implementing an SCons hierarchical build. The `exports` keyword argument allows the project configuration file to pass the `env` construction environment variable with the SCons sharing environments feature. The first simulation configuration will be added to the `workflow_configurations` list in Tutorial 01: Geometry.

11. Add the content below to the `SConstruct` file to add an empty default target list and to modify the project help message.

waves-tutorials/SConstruct

    101# Add default target list to help message
    102env.Default()  # Empty defaults list to avoid building all simulation targets by default
    103# Add aliases to help message so users know what build target options are available
    104# This must come *after* all expected Alias definitions and SConscript files.
    105env.ProjectHelp()

Because the WAVES tutorials contain a suite of simulations, it is useful to limit what SCons will build by default. To protect against running all simulations by default, create an empty default list. This will require that simulation targets are specified by name in the SCons build command. In addition to limiting the default target list, it is useful to print the list of default targets in the project help to remind developers what targets will build when no target is specified in the call to `scons`.

Simulation build workflows will typically involve many targets and tasks in a non-trivial execution order. The target file names may also be cumbersome to type when explicitly listing build targets in the SCons build command. For convenience, the WAVES tutorials simulation configurations will add a collector alias for the list of simulation targets with the SCons Alias feature. By convention, the WAVES tutorials match the alias name to the simulation subdirectory name. Tutorial 01: Geometry will introduce the first target alias, which will then populate the project help message displayed by the `scons -h` command option.

We previously added the `ProjectHelp` method to the construction environment using `waves.scons_extensions.WAVESEnvironment`. This points to `waves.scons_extensions.project_help()` which wraps two common calls to SCons Help that will append the following to the project help message accessed by `scons -h`:

- the command-line build options

- the default target list

- the project alias list

The help message recovers targets and aliases from the construction environment, so the `env.ProjectHelp` call must come after all `SConscript` and `Alias` method calls. Generally, it’s best to simply call this method as the final line in your project configuration.

Note

The project help message uses the conventional lowercase help option, `-h`. Most bash commands use this option to display the command’s help message corresponding to the software options. Due to this behavior, the SCons command options are displayed with the unconventional capitalized option, `-H`, as `scons -H`. The WAVES tutorials and documentation will not discuss the full range of SCons command options, so modsim developers are encouraged to read the SCons usage summary and SCons manpage to learn more about available build control options.

12. Explore the project help message

Linux/MacOS

    $ pwd
    /home/roppenheimer/waves-tutorials
    $ scons -h
    scons: Reading SConscript files ...
    Checking whether '/apps/abaqus/Commands/abq2024' program exists.../apps/abaqus/Commands/abq2024
    Checking whether '/usr/projects/ea/abaqus/Commands/abq2024' program exists...no
    Checking whether 'abq2024' program exists.../apps/abaqus/Commands/abq2024
    Checking whether 'abaqus' program exists...no
    scons: done reading SConscript files.
    Local Options:
      --build-dir=DIR           SCons build (variant) root directory. Relative or absolute path. (default: 'build')
      --unconditional-build     Boolean flag to force building of conditionally ignored targets, e.g. if the target's
                                   action program is missing and it would normally be ignored. (default: 'False')
      --print-build-failures    Print task *.stdout target file(s) on build failures. (default: 'False')
      --abaqus-command=COMMAND  Override for the Abaqus command. Repeat to specify more than one (default:
                                  ['/apps/abaqus/Commands/abq2024', '/usr/projects/ea/abaqus/Commands/abq2024', 'abq2024',
                                  'abaqus'])

    Default Targets:

    Target Aliases:

    Use scons -H for help about SCons built-in command-line options.

Windows

    PS > Get-Location

    Path
    ----
    C:\Users\roppenheimer\waves-tutorials

    PS > scons -h
    scons: Reading SConscript files ...
    Checking whether '/apps/abaqus/Commands/abq2024' program exists...no
    Checking whether '/usr/projects/ea/abaqus/Commands/abq2024' program exists...no
    Checking whether 'abq2024' program exists...C:\SIMULIA\Commands\abq2024.BAT
    Checking whether 'abaqus' program exists...C:\SIMULIA\Commands\abaqus.BAT
    scons: done reading SConscript files.
    Local Options:
      --build-dir=DIR           SCons build (variant) root directory. Relative or absolute path. (default: 'build')
      --unconditional-build     Boolean flag to force building of conditionally ignored targets, e.g. if the target's
                                   action program is missing and it would normally be ignored. (default: 'False')
      --print-build-failures    Print task *.stdout target file(s) on build failures. (default: 'False')
      --abaqus-command=COMMAND  Override for the Abaqus command. Repeat to specify more than one (default:
                                  ['/apps/abaqus/Commands/abq2024', '/usr/projects/ea/abaqus/Commands/abq2024', 'abq2024',
                                  'abaqus'])

    Default Targets:

    Target Aliases:

    Use scons -H for help about SCons built-in command-line options.

The text shown in the sample code block above is the project specific help message(s) added in the previous step. The `Default Targets:` and `Target Aliases:` lists will begin to populate in the following tutorial.