Metadata-Version: 2.4
Name: ai_ml_sdk_scenario_runner
Version: 0.11.0
Summary: ML SDK Scenario Runner
Author: Arm Limited
License: Apache-2.0
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSES/Apache-2.0.txt
Requires-Dist: numpy>=1.24
Dynamic: license-file

# ML SDK Scenario Runner

The Scenario Runner is an application that executes shader and neural network
graph workloads through Vulkan® or the ML extensions for Vulkan®. The Scenario
Runner acts as a validation and performance exploration vehicle. The Scenario
Runner also acts as a mechanism to define test-cases called scenarios in a
declarative way via a JSON description. The Scenario Runner can parse the JSON,
load the input stimulus that is described in the JSON, execute the scenario and
produce output artifacts.

## Cloning the repository

To clone the ML SDK Scenario Runner as a stand-alone repository, you can use
regular git clone commands. However, for better management of dependencies and
to ensure everything is placed in the appropriate directories, we recommend
using the `git-repo` tool to clone the repository as part of the ML SDK for
Vulkan® suite. [Repo tool](https://gerrit.googlesource.com/git-repo).

For a minimal build and to initialize only the Scenario Runner and its
dependencies, run:

```bash
repo init -u https://github.com/arm/ai-ml-sdk-manifest -g scenario-runner
```

Alternatively, to initialize the repo structure for the entire ML SDK for
Vulkan®, including the Scenario Runner, run:

```bash
repo init -u https://github.com/arm/ai-ml-sdk-manifest -g all
```

Once the repo is initialized, you can fetch the contents:

```bash
repo sync --no-clone-bundle
```

### Cloning on Windows®

To ensure nested submodules do not exceed the maximum long path length, you must
enable long paths on Windows®, and you must clone close to the root directory
or use a symlink. Make sure to use Git for Windows.

Using **PowerShell**:

```powershell
Set-ItemProperty -Path "HKLM:\SYSTEM\CurrentControlSet\Control\FileSystem" -Name "LongPathsEnabled" -Value 1
git config --global core.longpaths true
git --version # Ensure you are using Git for Windows, for example 2.50.1.windows.1
git clone <git-repo-tool-url>
python <path-to-git-repo>\git-repo\repo init -u <manifest-url> -g all
python <path-to-git-repo>\git-repo\repo sync --no-clone-bundle
```

Using **Git Bash**:

```bash
cmd.exe "/c reg.exe add \"HKLM\System\CurrentControlSet\Control\FileSystem"" /v LongPathsEnabled /t REG_DWORD /d 1 /f"
git config --global core.longpaths true
git --version # Ensure you are using the Git for Windows, for example 2.50.1.windows.1
git clone <git-repo-tool-url>
python <path-to-git-repo>/git-repo/repo init -u <manifest-url> -g all
python <path-to-git-repo>/git-repo/repo sync --no-clone-bundle
```

After the sync command completes successfully, you can find the ML SDK Scenario
Runner in `<repo_root>/sw/scenario-runner/`. You can also find all the
dependencies required by the ML SDK Scenario Runner in
`<repo_root>/dependencies/`.

## Building Scenario Runner from source

The build system must have:

- C/C++ 17 compiler: GCC or Clang on Linux, Clang on Darwin, or MSVC on
  Windows®.
- CMake 3.25 or later.
- Ninja 1.8.2 or later.
- Python 3.10 or later. Required python libraries for building are listed in
  `tooling-requirements.txt`.

The following dependencies are also needed:

- [Argument Parser for Modern C++](https://github.com/p-ranav/argparse).
- [DirectXShaderCompiler](https://github.com/microsoft/DirectXShaderCompiler).
- [glslang](https://github.com/KhronosGroup/glslang).
- [JSON for Modern C++](https://github.com/nlohmann/json).
- [SPIRV-Headers](https://github.com/KhronosGroup/SPIRV-Headers).
- [SPIRV-Tools](https://github.com/KhronosGroup/SPIRV-Tools).
- [Vulkan-Headers](https://github.com/KhronosGroup/Vulkan-Headers).
- [stb](https://github.com/nothings/stb) headers ``stb_image.h`` and ``stb_image_write.h`` (these are downloaded automatically during CMake configure)
- [GoogleTest](https://github.com/google/googletest). Optional, for testing.

For the preferred dependency versions see the manifest file.

## Building with the script

To build on Linux, run the following command:

```bash
SDK_PATH="path/to/sdk"
python3 ${SDK_PATH}/sw/scenario-runner/scripts/build.py -j $(nproc) \
    --flatbuffers-path ${SDK_PATH}/dependencies/flatbuffers \
    --argparse-path ${SDK_PATH}/dependencies/argparse \
    --json-path ${SDK_PATH}/dependencies/json \
    --vulkan-headers-path ${SDK_PATH}/dependencies/Vulkan-Headers \
    --glslang-path ${SDK_PATH}/dependencies/glslang \
    --dxc-path ${SDK_PATH}/dependencies/DirectXShaderCompiler \
    --spirv-headers-path ${SDK_PATH}/dependencies/SPIRV-Headers \
    --spirv-tools-path ${SDK_PATH}/dependencies/SPIRV-Tools \
    --vgf-lib-path ${SDK_PATH}/sw/vgf-lib \
    --gtest-path ${SDK_PATH}/dependencies/googletest
```

To build on Windows®, run the following command:

```powershell
$env:SDK_PATH="path\to\sdk"
$cores = [System.Environment]::ProcessorCount
python "$env:SDK_PATH\sw\scenario-runner\scripts\build.py" -j $cores  `
    --flatbuffers-path "$env:SDK_PATH\dependencies\flatbuffers" `
    --argparse-path "$env:SDK_PATH\dependencies\argparse" `
    --json-path "$env:SDK_PATH\dependencies\json" `
    --vulkan-headers-path "$env:SDK_PATH\dependencies\Vulkan-Headers" `
    --glslang-path "$env:SDK_PATH\dependencies\glslang" `
    --dxc-path "$env:SDK_PATH\dependencies\DirectXShaderCompiler" `
    --spirv-headers-path "$env:SDK_PATH\dependencies\SPIRV-Headers" `
    --spirv-tools-path "$env:SDK_PATH\dependencies\SPIRV-Tools" `
    --vgf-lib-path "$env:SDK_PATH\sw\vgf-lib" `
    --gtest-path "$env:SDK_PATH\dependencies\googletest"
```

To build on Darwin from a Repo tool checkout, first configure the
[LunarG Vulkan® SDK for Darwin](https://vulkan.lunarg.com/sdk/home#mac) environment
according to its documentation, then run:

```bash
export SDK_PATH=/absolute/path/to/ml-sdk
python3 "$SDK_PATH/sw/scenario-runner/scripts/build.py" \
    -j "$(sysctl -n hw.logicalcpu)"
```

At runtime, use the Vulkan® Loader to select either MoltenVK or KosmicKrisp and
enable the ML Emulation Layer for Vulkan®. The ML SDK's Darwin support is
experimental.

To cross compile for AArch64 architecture, you can add the following option:

```bash
SDK_PATH="path/to/sdk"
python3 $SDK_PATH/sw/scenario-runner/scripts/build.py -j $(nproc) \
    --flatbuffers-path ${SDK_PATH}/dependencies/flatbuffers \
    --argparse-path ${SDK_PATH}/dependencies/argparse \
    --json-path ${SDK_PATH}/dependencies/json \
    --vulkan-headers-path ${SDK_PATH}/dependencies/Vulkan-Headers \
    --glslang-path ${SDK_PATH}/dependencies/glslang \
    --dxc-path ${SDK_PATH}/dependencies/DirectXShaderCompiler \
    --spirv-headers-path ${SDK_PATH}/dependencies/SPIRV-Headers \
    --spirv-tools-path ${SDK_PATH}/dependencies/SPIRV-Tools \
    --vgf-lib-path ${SDK_PATH}/sw/vgf-lib \
    --gtest-path ${SDK_PATH}/dependencies/googletest \
    --target-platform aarch64
```

(Experimental) To cross compile for Android™, and build an APK package, you must target Android™, provide the Android™
CMake toolchain, and request the `apk` package type.

```bash
SDK_PATH="path/to/sdk"
ANDROID_NDK_HOME="path/to/android-ndk"
python3 ${SDK_PATH}/sw/scenario-runner/scripts/build.py -j $(nproc) \
    --target-platform android \
    --cmake-toolchain-for-android ${ANDROID_NDK_HOME}/build/cmake/android.toolchain.cmake \
    --flatbuffers-path ${SDK_PATH}/dependencies/flatbuffers \
    --argparse-path ${SDK_PATH}/dependencies/argparse \
    --json-path ${SDK_PATH}/dependencies/json \
    --vulkan-headers-path ${SDK_PATH}/dependencies/Vulkan-Headers \
    --glslang-path ${SDK_PATH}/dependencies/glslang \
    --dxc-path ${SDK_PATH}/dependencies/DirectXShaderCompiler \
    --spirv-headers-path ${SDK_PATH}/dependencies/SPIRV-Headers \
    --spirv-tools-path ${SDK_PATH}/dependencies/SPIRV-Tools \
    --vgf-lib-path ${SDK_PATH}/sw/vgf-lib \
    --package-type apk
```

The APK build uses the `gradle` command, so Gradle 8.4 or later must be
available on your `PATH`. The Android™ SDK installation pointed to by
`ANDROID_HOME` should include `build-tools;34.0.0` and
`platforms;android-34`, or other compatible versions.

To build with RenderDoc support, add `--enable-rdoc`. If RenderDoc is
installed outside the default search locations, also pass
`--renderdoc-path <renderdoc-install-root>`.

To enable and run tests, use the `--test` flag. To lint the tests, use the
`--lint` flag. To enable tests and documentation building python dependencies
must be installed:

```bash
pip install -r requirements.txt
pip install -r tooling-requirements.txt
```

To build the documentation, use the `--doc` flag. To build the documentation,
you must have `sphinx` and `doxygen` installed on your machine.

You can install the build artifacts for this project into a specified location.
To install the build artifacts, pass the `--install` option with the required
path.

To create an archive with the build artifacts, you must add `--package-type`. The
archive is stored in the provided location.

For more command line options, consult the program help:

```bash
python3 $SDK_PATH/sw/scenario-runner/scripts/build.py --help
```

## Usage

To run a scenario file, use the following command:

```bash
./scenario-runner --scenario ${SCENARIO_JSON_FILE}
```

Where:

- `--scenario`: File to load the scenario from. The file must be in JSON format.
  If the resources in the SCENARIO_JSON_FILE are not specified with absolute
  paths, their relative paths will be resolved against the parent directory of
  the SCENARIO_JSON_FILE.

For more details, see the help output:

```bash
./scenario-runner --help
```

For the Android APK package, install the generated APK and start the foreground
service with `adb`:

```bash
adb install -r build/scenario-runner-debug.apk

adb shell am start-foreground-service \
    -n com.arm.ai_ml_sdk_scenario_runner/.Main \
    --esa args --scenario,/data/user/0/com.arm.ai_ml_sdk_scenario_runner/scenario.json
```

## PyPI

The ML SDK Scenario Runner is available on PyPI as the [ai-ml-sdk-scenario-runner](https://pypi.org/project/ai-ml-sdk-scenario-runner) package.

Install the published package:

```bash
pip install ai-ml-sdk-scenario-runner
```

## Python bindings

Python bindings are available through the `scenario_runner` package:

```python
import numpy as np
import scenario_runner

builder = scenario_runner.ScenarioBuilder()
image_id = builder.add_image(
    [1, 2, 2, 1],
    scenario_runner.Format.R8Uint,
    is_input=True,
    is_sampled=True,
    tiling=scenario_runner.Tiling.Linear,
)
scenario = builder.build()

image = np.array([1, 2, 3, 4], dtype=np.uint8).reshape(1, 2, 2, 1)
scenario.upload(image_id, image)
scenario.run()
result = scenario.download(image_id)
```

## Install from source with pip

From the repository root, install the package with:

```bash
pip install .
```

This command builds the host executable and Python bindings, then installs the
resulting Python package. Ensure the prerequisites listed in
[Building Scenario Runner from source](#building-scenario-runner-from-source)
are installed and the repository dependencies have been initialized.

## Graph Shape Inference

Graph shape inference is experimental. A scenario requests graph shape inference
by marking graph output tensor dimensions with `?`. Fully shaped graph outputs
do not request shape inference. Input tensors must be fully shaped, and Scenario
Runner does not inspect SPIR-V™ modules to detect unshaped graphs automatically.

## Known Limitations

- Resources created with `Optimal` tiling cannot be used with memory aliasing.
- HLSL lacks support for VK_ARM_tensor at the moment, so tensor resources cannot be used with HLSL shaders.
- For graph shape inference, partially specified output shapes are not supported
  as constraints. Tensor dimensions must be either unranked or ranked with all
  dynamic shapes. Mixed dimensions such as `[1, "?", 2, "?"]` are not supported.
  Inferred output shapes replace the declared dimensions.
- A graph resource requiring shape inference cannot be reused by dispatches
  with different interface shapes. Declare a separate graph resource for each
  shape.

## License

The ML SDK Scenario Runner is distributed under the software licenses in
LICENSES directory.

## Trademark notice

Arm® is a registered trademark of Arm Limited (or its subsidiaries) in the US
and/or elsewhere.

Khronos® and Vulkan® are registered trademarks, and SPIR-V™ is a trademark of
[The Khronos Group Inc.](https://www.khronos.org/legal/trademarks/).
