Skip to main content
`Ctrl`+`K`

# Tutorial: Task Definition Reuse

Warning

This tutorial is in a draft outline state. It is provided as reference and will be fleshed out in future releases. Be sure to check back in on this tutorial or watch the Changelog for updates!

This tutorial demonstrates one solution to re-using task definitions in multiple workflows. While the solution can be implemented with per-task granularity, the task definitions in this tutorial are split into blocks to suite the parameterization workflows of Tutorial 07: Cartesian Product and Tutorial: Mesh Convergence.

The presented solution comes with significant restrictions to directory organization and challenges to target node identification. First, due to the way SCons constructs variant (build) directory directed graphs, the re-usable portions of the workflow must be co-located with the calling file or found in a subdirectory. The choice of organization has implications for the build directory organization options, too. For example, the two possible directory organization may look like one of

    project directory/
    |-- SConscruct
    |-- workflow1
    |-- workflow2
    |-- common
    `-- build directory/
        |-- workflow1 directory/
        |   |-- workflow1 output
        |   |-- common1 output
        `-- workflow2 directory*
            |-- workflow2 output
            `-- common2 output

    project directory/
    |-- SConscruct
    |-- workflow1
    |-- workflow2
    |-- common directory/
    |   `-- common
    `-- build directory/
        |-- workflow1 directory/
        |   |-- workflow1 output
        |   `-- common directory/
        |       `-- common1 output
        `-- workflow2 directory/
            |-- workflow2 output
            `-- common directory/
                `-- common2 output

Second, if a target from the parent file is used as a source in the common file, SCons will recognize the build path correctly when provided with the target file base name, but it may not associate the source and target nodes correctly. This results in an incorrectly assembled directed graph and race conditions in the task execution. The target/source association may be made explicit with the SCons FindFile function. While this works, it is not as straightforward as simply specifying the file basename, as implemented in the core tutorials. This second restriction does not affect the current tutorial, but will affect workflows similar to the Tutorial: Mesh Convergence.

## References

- SCons FindFile [32]

## Environment

SCons and WAVES can be installed in a Conda environment with the Conda package manager. See the Conda installation and Conda environment management documentation for more details about using Conda.

Note

The SALib and numpy versions may not need to be this strict for most tutorials. However, Tutorial: Sensitivity Study uncovered some undocumented SALib version sensitivity to numpy surrounding the numpy v2 rollout.

1. Create the tutorials environment if it doesn’t exist

   Linux/MacOS

       $ conda create --name waves-tutorial-env --channel conda-forge waves 'scons>=4.6' matplotlib pandas pyyaml xarray seaborn 'numpy>=2' 'salib>=1.5.1' pytest

   Windows

       PS > conda create --name waves-tutorial-env --channel conda-forge waves scons matplotlib pandas pyyaml xarray seaborn numpy salib pytest

2. Activate the environment

   Linux/MacOS

       $ conda activate waves-tutorial-env

   Windows

       PS > conda activate waves-tutorial-env

Some tutorials require additional third-party software that is not available for the Conda package manager. This software must be installed separately and either made available to SConstruct by modifying your system’s `PATH` or by modifying the SConstruct search paths provided to the `waves.scons_extensions.add_program()` method.

## Directory Structure

3. Create and change to a new project root directory to house the tutorial files if you have not already done so. For example

Linux/MacOS

    $ mkdir -p ~/waves-tutorials
    $ cd ~/waves-tutorials
    $ pwd
    /home/roppenheimer/waves-tutorials

Windows

    PS > New-Item $HOME\waves-tutorials -ItemType "Directory"
    PS > Set-Location $HOME\waves-tutorials
    PS > Get-Location

    Path
    ----
    C:\Users\roppenheimer\waves-tutorials

Note

If you skipped any of the previous tutorials, run the following commands to create a copy of the necessary tutorial files.

    $ pwd
    /home/roppenheimer/waves-tutorials
    $ waves fetch --overwrite --tutorial 12 && mv tutorial_12_archival_SConstruct SConstruct
    WAVES fetch
    Destination directory: '/home/roppenheimer/waves-tutorials'

4. Download and copy the `tutorial_12_archival` file to a new file named `tutorial_task_reuse` with the WAVES Command-Line Utility fetch subcommand.

    $ pwd
    /home/roppenheimer/waves-tutorials
    $ waves fetch --overwrite tutorials/tutorial_12_archival && cp tutorial_12_archival tutorial_task_reuse
    WAVES fetch
    Destination directory: '/home/roppenheimer/waves-tutorials'

## SConscript

A `diff` against the `tutorial_12_archival` file from Tutorial 12: Data Archival is included below to help identify the changes made in this tutorial.

waves-tutorials/tutorial_task_reuse

    --- /home/runner/work/waves/waves/build/docs/tutorials_tutorial_12_archival
    +++ /home/runner/work/waves/waves/build/docs/tutorials_tutorial_task_reuse
    @@ -27,8 +27,10 @@
     # Simulation variables
     build_directory = pathlib.Path(Dir(".").abspath)
     workflow_name = build_directory.name
    -workflow_configuration = [env["project_configuration"], workflow_name]
     parameter_study_file = build_directory / "parameter_study.h5"
    +geometry_configuration = "rectangle_geometry_partition.scons"
    +simulation_configuration = "rectangle_mesh_solverprep_solve_extract.scons"
    +workflow_configuration = [env["project_configuration"], workflow_name, geometry_configuration, simulation_configuration]

     # Collect the target nodes to build a concise alias for all targets
     workflow = []
    @@ -53,115 +55,33 @@

         # Comment used in tutorial code snippets: marker-4

    -    # Geometry
    -    workflow.extend(
    -        env.AbaqusJournal(
    -            target=[set_name / "rectangle_geometry.cae", set_name / "rectangle_geometry.jnl"],
    -            source=["#/modsim_package/abaqus/rectangle_geometry.py"],
    -            subcommand_options="--width ${width} --height ${height}",
    -            **simulation_variables,
    -        )
    +    # Geometry, Partition
    +    workflow, datacheck = env.SConscript(
    +        geometry_configuration,
    +        variant_dir=set_name.name,
    +        exports={
    +            "env": env,
    +            "simulation_variables": simulation_variables,
    +            "workflow": workflow,
    +            "datacheck": datacheck,
    +        },
    +        duplicate=False,
         )

    -    # Partition
    -    workflow.extend(
    -        env.AbaqusJournal(
    -            target=[set_name / "rectangle_partition.cae", set_name / "rectangle_partition.jnl"],
    -            source=["#/modsim_package/abaqus/rectangle_partition.py", set_name / "rectangle_geometry.cae"],
    -            subcommand_options="--width ${width} --height ${height}",
    -            **simulation_variables,
    -        )
    +    # Mesh, SolverPrep, Abaqus Solve, Extract Abaqus
    +    workflow, datacheck = env.SConscript(
    +        simulation_configuration,
    +        variant_dir=set_name.name,
    +        exports={
    +            "env": env,
    +            "simulation_variables": simulation_variables,
    +            "workflow": workflow,
    +            "datacheck": datacheck,
    +        },
    +        duplicate=False,
         )

    -    # Mesh
    -    workflow.extend(
    -        env.AbaqusJournal(
    -            target=[
    -                set_name / "rectangle_mesh.inp",
    -                set_name / "rectangle_mesh.cae",
    -                set_name / "rectangle_mesh.jnl",
    -            ],
    -            source=["#/modsim_package/abaqus/rectangle_mesh.py", set_name / "rectangle_partition.cae"],
    -            subcommand_options="--global-seed ${global_seed}",
    -            **simulation_variables,
    -        )
    -    )
    -
    -    # SolverPrep
    -    copy_source_list = [
    -        "#/modsim_package/abaqus/rectangle_compression.inp.in",
    -        "#/modsim_package/abaqus/assembly.inp",
    -        "#/modsim_package/abaqus/boundary.inp",
    -        "#/modsim_package/abaqus/field_output.inp",
    -        "#/modsim_package/abaqus/materials.inp",
    -        "#/modsim_package/abaqus/parts.inp",
    -        "#/modsim_package/abaqus/history_output.inp",
    -    ]
    -    workflow.extend(
    -        env.CopySubstfile(
    -            copy_source_list,
    -            substitution_dictionary=env.SubstitutionSyntax(simulation_variables),
    -            build_subdirectory=set_name,
    -        )
    -    )
    -
    -    # Comment used in tutorial code snippets: marker-5
    -
    -    # Abaqus Solve
    -    solve_source_list = [
    -        set_name / "rectangle_compression.inp",
    -        set_name / "assembly.inp",
    -        set_name / "boundary.inp",
    -        set_name / "field_output.inp",
    -        set_name / "materials.inp",
    -        set_name / "parts.inp",
    -        set_name / "history_output.inp",
    -        set_name / "rectangle_mesh.inp",
    -    ]
    -
    -    datacheck.extend(
    -        env.AbaqusSolver(
    -            target=[
    -                set_name / "rectangle_compression_DATACHECK.odb",
    -                set_name / "rectangle_compression_DATACHECK.dat",
    -                set_name / "rectangle_compression_DATACHECK.msg",
    -                set_name / "rectangle_compression_DATACHECK.com",
    -                set_name / "rectangle_compression_DATACHECK.prt",
    -                set_name / "rectangle_compression_DATACHECK.023",
    -                set_name / "rectangle_compression_DATACHECK.mdl",
    -                set_name / "rectangle_compression_DATACHECK.sim",
    -                set_name / "rectangle_compression_DATACHECK.stt",
    -            ],
    -            source=solve_source_list,
    -            job="rectangle_compression_DATACHECK",
    -            program_options="-double both -datacheck",
    -        )
    -    )
    -
    -    workflow.extend(
    -        env.AbaqusSolver(
    -            target=[
    -                set_name / "rectangle_compression.odb",
    -                set_name / "rectangle_compression.dat",
    -                set_name / "rectangle_compression.msg",
    -                set_name / "rectangle_compression.com",
    -                set_name / "rectangle_compression.prt",
    -                set_name / "rectangle_compression.sta",
    -            ],
    -            source=solve_source_list,
    -            job="rectangle_compression",
    -            program_options="-double both",
    -        )
    -    )
    -
    -    # Extract Abaqus
    -    extract_source_list = [set_name / "rectangle_compression.odb"]
    -    workflow.extend(
    -        env.AbaqusExtract(
    -            target=[set_name / "rectangle_compression.h5"],
    -            source=extract_source_list,
    -        )
    -    )
    +# Comment used in tutorial code snippets: marker-5

     # Comment used in tutorial code snippets: marker-6

5. Create a new file named `rectangle_geometry_partition.scons` from the contents below

waves-tutorials/rectangle_geometry_partition.scons

     1#! /usr/bin/env python
     2"""Rectangle model's Geometry and Partition task definitions
     3
     4Requires the following ``SConscript(..., exports={})``
     5
     6* ``env`` - The SCons construction environment with the following required keys
     7
     8
     9* ``simulation_variables`` - The dictionary of simulation variables with the following required keys
    10
    11  * ``'width'`` - The rectangle model width
    12  * ``'height'`` - The rectangle model height
    13
    14* ``workflow`` - list of targets in the main workflow
    15* ``datacheck`` - list of targets in the datacheck workflow
    16
    17Returns the following variables
    18
    19* ``workflow`` - updated list of targets in the main workflow
    20* ``datacheck`` - updated list of targets in the datacheck workflow
    21
    22Example usage:
    23
    24.. code-block::
    25
    26   import pathlib
    27
    28   env = Environment()
    29   project_dir = pathlib.Path(Dir(".").abspath)
    30
    31   workflow = []
    32   datacheck = []
    33
    34   nominal_workflow, nominal_datacheck = SConscript(
    35       "rectangle_geometry_partition.scons",
    36       exports={
    37           "env": env,
    38           "simulation_variables": simulation_variables,
    39           "workflow": workflow,
    40           "datacheck": datacheck
    41       },
    42       duplicate=False)
    43
    44   for set_name in [f"parmameter_set{number}" for number in range(4)]
    45       parameters_workflow, parameter_datacheck = SConscript(
    46           "rectangle_geometry_partition.scons",
    47           variant_dir=set_name,
    48           exports={
    49               "env": env,
    50               "simulation_variables": simulation_variables,
    51               "workflow": workflow,
    52               "datacheck": datacheck
    53           },
    54           duplicate=False)
    55"""
    56
    57import pathlib
    58
    59# Inherit the parent construction environment
    60Import(["env", "simulation_variables", "workflow", "datacheck"])
    61
    62# Simulation variables
    63build_directory = pathlib.Path(Dir(".").abspath)
    64
    65# Geometry
    66workflow.extend(
    67    env.AbaqusJournal(
    68        target=["rectangle_geometry.cae", "rectangle_geometry.jnl"],
    69        source=["#/modsim_package/abaqus/rectangle_geometry.py"],
    70        subcommand_options="--width ${width} --height ${height}",
    71        **simulation_variables,
    72    )
    73)
    74
    75# Partition
    76workflow.extend(
    77    env.AbaqusJournal(
    78        target=["rectangle_partition.cae", "rectangle_partition.jnl"],
    79        source=["#/modsim_package/abaqus/rectangle_partition.py", "rectangle_geometry.cae"],
    80        subcommand_options="--width ${width} --height ${height}",
    81        **simulation_variables,
    82    )
    83)
    84
    85Return(["workflow", "datacheck"])

6. Create a new file named `rectangle_mesh_solverprep_solve_extract.scons` from the contents below

waves-tutorials/rectangle_mesh_solverprep_solve_extract.scons

      1#! /usr/bin/env python
      2"""Rectangle model's Geometry and Partition task definitions
      3
      4Requires the following ``SConscript(..., exports={})``
      5
      6* ``env`` - The SCons construction environment with the following required keys
      7
      8
      9* ``simulation_variables`` - The dictionary of simulation variables with the following required keys
     10
     11  * ``'global_seed'`` - The rectangle model global seed size
     12  * ``'displacement'`` - The rectangle model uniaxial displacement
     13
     14* ``workflow`` - list of targets in the main workflow
     15* ``datacheck`` - list of targets in the datacheck workflow
     16
     17Returns the following variables
     18
     19* ``workflow`` - updated list of targets in the main workflow
     20* ``datacheck`` - updated list of targets in the datacheck workflow
     21
     22Example usage:
     23
     24.. code-block::
     25
     26   import pathlib
     27
     28   env = waves.scons_extensions.WAVESEnvironment()
     29   project_dir = pathlib.Path(Dir(".").abspath)
     30
     31   workflow = []
     32   datacheck = []
     33
     34   nominal_workflow, nominal_datacheck = SConscript(
     35       "rectangle_mesh_solverpep_solve_extract.scons",
     36       exports={
     37           "env": env,
     38           "simulation_variables": simulation_variables,
     39           "workflow": workflow,
     40           "datacheck": datacheck
     41       },
     42       duplicate=False)
     43
     44   for set_name in [f"parmameter_set{number}" for number in range(4)]
     45       parameters_workflow, parameter_datacheck = SConscript(
     46           "rectangle_mesh_solverpep_solve_extract.scons",
     47           variant_dir=set_name,
     48           exports=["env", "simulation_variables", "workflow", "datacheck")
     49           duplicate=False)
     50"""
     51import pathlib
     52
     53
     54# Inherit the parent construction environment
     55Import(["env", "simulation_variables", "workflow", "datacheck"])
     56
     57# Simulation variables
     58build_directory = pathlib.Path(Dir(".").abspath)
     59
     60# Mesh
     61workflow.extend(
     62    env.AbaqusJournal(
     63        target=["rectangle_mesh.inp", "rectangle_mesh.cae", "rectangle_mesh.jnl"],
     64        source=["#/modsim_package/abaqus/rectangle_mesh.py", "rectangle_partition.cae"],
     65        subcommand_options="--global-seed ${global_seed}",
     66        **simulation_variables,
     67    )
     68)
     69
     70# SolverPrep
     71copy_source_list = [
     72    "#/modsim_package/abaqus/rectangle_compression.inp.in",
     73    "#/modsim_package/abaqus/assembly.inp",
     74    "#/modsim_package/abaqus/boundary.inp",
     75    "#/modsim_package/abaqus/field_output.inp",
     76    "#/modsim_package/abaqus/materials.inp",
     77    "#/modsim_package/abaqus/parts.inp",
     78    "#/modsim_package/abaqus/history_output.inp",
     79]
     80workflow.extend(
     81    env.CopySubstfile(
     82        copy_source_list,
     83        substitution_dictionary=env.SubstitutionSyntax(simulation_variables),
     84    )
     85)
     86
     87# Abaqus Solve
     88solve_source_list = [
     89    "rectangle_compression.inp",
     90    "assembly.inp",
     91    "boundary.inp",
     92    "field_output.inp",
     93    "materials.inp",
     94    "parts.inp",
     95    "history_output.inp",
     96    "rectangle_mesh.inp",
     97]
     98
     99datacheck.extend(
    100    env.AbaqusSolver(
    101        target=[
    102            "rectangle_compression_DATACHECK.odb",
    103            "rectangle_compression_DATACHECK.dat",
    104            "rectangle_compression_DATACHECK.msg",
    105            "rectangle_compression_DATACHECK.com",
    106            "rectangle_compression_DATACHECK.prt",
    107            "rectangle_compression_DATACHECK.023",
    108            "rectangle_compression_DATACHECK.mdl",
    109            "rectangle_compression_DATACHECK.sim",
    110            "rectangle_compression_DATACHECK.stt",
    111        ],
    112        source=solve_source_list,
    113        job="rectangle_compression_DATACHECK",
    114        program_options="-double both -datacheck",
    115    )
    116)
    117
    118workflow.extend(
    119    env.AbaqusSolver(
    120        target=[
    121            "rectangle_compression.odb",
    122            "rectangle_compression.dat",
    123            "rectangle_compression.msg",
    124            "rectangle_compression.com",
    125            "rectangle_compression.prt",
    126            "rectangle_compression.sta",
    127        ],
    128        source=solve_source_list,
    129        job="rectangle_compression",
    130        program_options="-double both",
    131    )
    132)
    133
    134# Extract Abaqus
    135extract_source_list = ["rectangle_compression.odb"]
    136workflow.extend(
    137    env.AbaqusExtract(
    138        target=["rectangle_compression.h5"],
    139        source=extract_source_list,
    140    )
    141)
    142
    143Return(["workflow", "datacheck"])

## SConstruct

A `diff` against the `SConstruct` file from Tutorial 12: Data Archival is included below to help identify the changes made in this tutorial.

waves-tutorials/SConstruct

    --- /home/runner/work/waves/waves/build/docs/tutorials_tutorial_12_archival_SConstruct
    +++ /home/runner/work/waves/waves/build/docs/tutorials_tutorial_task_reuse_SConstruct
    @@ -61,6 +61,7 @@
         print_build_failures=GetOption("print_build_failures"),
         abaqus_commands=GetOption("abaqus_command"),
         TARFLAGS="-c -j",
    +    TARSUFFIX=".tar.bz2",
     )

     # Conditionally print failed task *.stdout files
    @@ -121,6 +122,7 @@
         "tutorial_09_post_processing",
         "tutorial_11_regression_testing",
         "tutorial_12_archival",
    +    "tutorial_task_reuse",
     ]
     for workflow in workflow_configurations:
         build_dir = env["variant_dir_base"] / workflow

## Build Targets

7. Build the new targets

    $ pwd
    /home/roppenheimer/waves-tutorials
    $ scons tutorial_task_reuse --jobs=4

## Output Files

    $ pwd
    /home/roppenheimer/waves-tutorials
    $ tree build/tutorial_task_reuse/parameter_set0/
    build/tutorial_task_reuse/parameter_set0/
    |-- abaqus.rpy
    |-- abaqus.rpy.1
    |-- abaqus.rpy.2
    |-- assembly.inp
    |-- boundary.inp
    |-- field_output.inp
    |-- history_output.inp
    |-- materials.inp
    |-- parts.inp
    |-- rectangle_compression.inp
    |-- rectangle_compression.inp.in
    |-- rectangle_compression_DATACHECK.023
    |-- rectangle_compression_DATACHECK.com
    |-- rectangle_compression_DATACHECK.dat
    |-- rectangle_compression_DATACHECK.mdl
    |-- rectangle_compression_DATACHECK.msg
    |-- rectangle_compression_DATACHECK.odb
    |-- rectangle_compression_DATACHECK.prt
    |-- rectangle_compression_DATACHECK.sim
    |-- rectangle_compression_DATACHECK.stdout
    |-- rectangle_compression_DATACHECK.stt
    |-- rectangle_geometry.cae
    |-- rectangle_geometry.jnl
    |-- rectangle_geometry.stdout
    |-- rectangle_mesh.cae
    |-- rectangle_mesh.inp
    |-- rectangle_mesh.jnl
    |-- rectangle_mesh.stdout
    |-- rectangle_partition.cae
    |-- rectangle_partition.jnl
    `-- rectangle_partition.stdout

    0 directories, 30 files