Metadata-Version: 2.2
Name: gaia-ml
Version: 0.6.0
Summary: GAIA-ML ONNX-to-AIE project generator
Author: I. Xiotidis
Maintainer: NGT WP2.1 Group
License: MIT
Project-URL: Homepage, https://gitlab.cern.ch/atlas-nextgen-wp21/aie/gaia-ml
Project-URL: Documentation, https://gitlab.cern.ch/atlas-nextgen-wp21/aie/gaia-ml/-/blob/main/README.md
Project-URL: Repository, https://gitlab.cern.ch/atlas-nextgen-wp21/aie/gaia-ml
Project-URL: Issues, https://gitlab.cern.ch/atlas-nextgen-wp21/aie/gaia-ml/-/issues
Keywords: aie,amd,cmake,machine-learning,onnx,vitis
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Code Generators
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: jinja2>=3.1
Requires-Dist: numpy>=1.22
Requires-Dist: onnx>=1.14

# GAIA-ML

Global AI-Engine Architecture - ML (GAIA-ML) is a software tool hoping to automatically transform ML algorithms into AMD AI-Engine (AIE) optimal implementations. GAIA-ML is developed within the ATLAS Global Trigger group, as the ML layers implemented are designed to provide optimal computation for low latency and high-throughput environments

Main developer: I. Xiotidis.

GAIA-ML is developed by the NGT WP2.1 Group.

License: MIT.

## Framework infrastructure

GAIA-ML follows a layers approach where each layer of the ML architecture is coded in a single AIE kernel. Having each layer coded in a single layer doesn't mean that the final result will be deployed in different AIE tiles (parametric graph implementation).

<!-- GAIA_STATUS_START -->
## Project Status

This section is generated from the repository contents. Refresh it with:

```bash
cmake -S . -B build
cmake --build build --target readme-status
```

or directly:

```bash
python3 tools/update_readme_status.py
```

### Layers

| Layer | Submodule | Branch | SHA | CMake | Stream I/O | Kernel headers | Local graphs |
| --- | --- | --- | --- | --- | --- | --- | --- |
| activation | yes | master | bc7efd9 | yes | no | activation.hpp | - |
| conv | yes | cnn_v2 | 794c567 | yes | yes | activation_policy.hpp, conv2d.hpp, conv2d_depthwise.hpp | convGraph.cpp, depthwiseGraph.cpp |
| dense | yes | gaia_int | c2ae42f | yes | yes | activation_policy.hpp, dense.hpp, dense_aux.hpp | denseAuxGraph.cpp, denseGraph.cpp |
| tensor_op | yes | master | 39e64de | yes | yes | flatten_chw.hpp | - |

### CMake Targets

```text
aie
aie-conv2d
aie-dense
aie-dense-aux
aie-depthwise
aiesim
aiesim-conv2d
aiesim-dense
aiesim-dense-aux
aiesim-depthwise
data
data-conv2d
data-dense
data-depthwise
gaia_activation_kernel
gaia_conv_kernel
gaia_dense_kernel
gaia_kernels
gaia_tensor_op_kernel
layer_aie
layer_aiesim
layer_data
layer_x86
layer_x86sim
readme-status
x86
x86-conv2d
x86-dense
x86-dense-aux
x86-depthwise
x86sim
x86sim-conv2d
x86sim-dense
x86sim-dense-aux
x86sim-depthwise
```

### Repository Layout

```text
.
|-- examples
|   |-- conv_flatten_dense_onnx_generate.py
|   `-- dense_onnx_generate.py
|-- layers
|   |-- activation
|   |   |-- Kernel
|   |   |   `-- include
|   |   |-- .gitignore
|   |   |-- CMakeLists.txt
|   |   `-- README.md
|   |-- conv
|   |   |-- Graph
|   |   |   |-- convGraph.cpp
|   |   |   |-- convGraph.hpp
|   |   |   |-- depthwiseGraph.cpp
|   |   |   `-- depthwiseGraph.hpp
|   |   |-- Kernel
|   |   |   |-- include
|   |   |   `-- src
|   |   |-- testVectors
|   |   |   `-- makeDummyData.py
|   |   |-- .gitignore
|   |   |-- CMakeLists.txt
|   |   `-- README.md
|   |-- dense
|   |   |-- Graph
|   |   |   |-- denseAuxGraph.cpp
|   |   |   |-- denseAuxGraph.hpp
|   |   |   |-- denseGraph.cpp
|   |   |   `-- denseGraph.hpp
|   |   |-- Kernel
|   |   |   |-- include
|   |   |   `-- src
|   |   |-- testVectors
|   |   |   `-- makeDummyData.py
|   |   |-- .gitignore
|   |   |-- CMakeLists.txt
|   |   |-- notes.md
|   |   `-- README.md
|   `-- tensor_op
|       |-- Kernel
|       |   |-- include
|       |   `-- src
|       |-- CMakeLists.txt
|       `-- README.md
|-- python
|   |-- gaia_ml
|   |   |-- __init__.py
|   |   |-- _version.py
|   |   |-- cli.py
|   |   |-- limits.py
|   |   |-- model_ir.py
|   |   |-- onnx_frontend.py
|   |   `-- project.py
|   `-- gaia_ml.egg-info
|       |-- dependency_links.txt
|       |-- entry_points.txt
|       |-- PKG-INFO
|       |-- requires.txt
|       |-- SOURCES.txt
|       `-- top_level.txt
|-- tools
|   `-- update_readme_status.py
|-- .gitignore
|-- .gitmodules
|-- CMakeLists.txt
|-- pyproject.toml
|-- README.md
`-- setup.cfg
```
<!-- GAIA_STATUS_END -->

## CMake hierarchy

Each layer repository can be configured and built on its own. In that mode, the
layer-local graph is used for `x86`, `aie`, `x86sim`, and `aiesim` targets.

From the top-level package, layer repositories are added as kernel providers.
The exported targets are:

```text
gaia_activation_kernel
gaia_conv_kernel
gaia_dense_kernel
gaia_kernels
```

Enable `GAIA_BUILD_LAYER_GRAPHS=ON` only when you also want the top-level build
to expose the layer-local smoke-test graph targets.

## Notes

GAIA-ML v1.0 supports stream I/O only. Layer kernels use
`input_stream<int32>` and `output_stream<int32>` ports so the
top-level graph can be built around a single low-latency dataflow contract.

The current convolution stream kernels keep the existing frame-based compute
path internally by reading one streamed frame into local AIE memory before
computing. This preserves the stream interface for v1.0 while leaving room for
a later line-buffer implementation that can reduce first-output latency.

Activations are header-only compile-time policies in v1.0. Layers default to
`gaia::activation::identity` when the shared activation header is not on the
include path, and the top-level package exposes `activation.hpp` so graphs can
fuse policies such as `linear`, `relu`, `sigmoid`, `tanh`, or `softmax` into the
producer kernel instead of adding a separate activation tile.

## Python Automation

The first automation scaffold is available as a pip-installable package:

```bash
pip install -e .
gaia_ml version
gaia_ml inspect model.onnx
gaia_ml generate model.onnx --output build/generated_model --strategy throughput
```

As of `gaia_ml` v0.5.0, generated project structure is emitted from Jinja2
templates packaged under `gaia_ml/templates/aie`. Backend Python code now builds
the graph/layer context and renders the project files from templates, so changes
to generated `Graph/`, `CMakeLists.txt`, and generated README layout no longer
need to be hard-coded directly into `project.py`.

As of `gaia_ml` v0.6.0, generated kernels keep the same stream interface but
avoid full input-frame caching where possible. Depthwise Conv uses a K-row line
buffer per channel; generic Conv and 1x1 pointwise Conv stream input rows and
accumulate output partials; Dense and Dense-CHW stream inputs directly into
output accumulators. This preserves the existing CHW padded stream contract
while reducing local input storage and preparing the backend for more aggressive
overlap.

Project generation supports two lowering strategies:

```text
throughput  Use more AIE tiles and preserve layer-level graph pipelining.
compact     Fuse supported patterns to minimize tile count.
```

`throughput` is the default because AIE latency often benefits from multiple
tiles running concurrently. `compact` is useful when fitting into the smallest
number of tiles matters more than first-output latency.

The current generator parses ONNX Conv, depthwise Conv, 1x1 pointwise Conv,
Flatten, and Dense/Gemm-style blocks, runs ONNX shape inference, checks what
can be emitted, and creates a self-contained AIE project with `Graph/`,
`Kernel/`, `weights/`, `testVectors/`, and `CMakeLists.txt`. Generation emits
one packed-output tile per supported compute block and chains them in model
order, unless a low-latency fused pattern is available:

```text
input stream -> layer0 -> layer1 -> ... -> output stream
```

`gaia_ml inspect` always reports optimization suggestions, including patterns
that are not generated yet. The CLI uses colored log levels for info, warning,
critical warning, and error messages so compatibility problems and latency
recommendations are easier to scan.

The current `int32` backend pads dense vectors and convolution rows to 8-lane
chunks. Dense blocks are rejected if either side exceeds 128 padded lanes.
Convolution blocks are currently single-tile only: input spatial dimensions are
limited to 32x32, kernels to 8x8, padded input/output rows to 128 lanes, and
padded frame buffers to the current local-memory guardrails. Larger layers need
an explicit tiling strategy before generation.

`Flatten` is represented in the Python IR as a tensor operation. When possible,
it is fused away: `Conv -> Flatten -> Dense` lowers to a `packed_dense_chw`
consumer that reads the padded CHW stream directly and skips row padding without
an extra AIE tile. The `tensor_op` layer also provides an explicit
`flatten_chw` kernel for future graph cases where a standalone reshuffle tile is
actually required. ONNX depthwise Conv lowers to `packed_depthwise_conv2d`, and
ONNX 1x1 Conv lowers to a specialized `packed_pointwise_conv2d` kernel. Weights
are emitted as headers under `weights/`, and the generated
`testVectors/reference_output.csv` contains the expected end-to-end inference
stream for the generated test input.
The first low-latency fused lowering is
`DepthwiseConv -> 1x1 Conv -> Flatten -> Dense(1)`, which emits
`packed_dw_pw_dense_scalar`. The generic kernels are still emitted in generated
projects so each kernel can continue to be developed and tested independently.
Generated graphs write simulator output to `testVectors/model_output.csv`; after
hardware AIE simulation this appears under
`aiesimulator_output/testVectors/model_output.csv`, while the expected output
remains in `testVectors/reference_output.csv`.
The generated project can be built with:

```bash
cmake -S build/generated_model -B build/generated_model/build
cmake --build build/generated_model/build --target x86
cmake --build build/generated_model/build --target x86sim
```

For now, project emission supports sequential Conv, Flatten, and Dense/Gemm
models with optional supported activations and zero/no bias. Branching, general
transpose/concat/split reshuffles, softmax fusion, and larger-than-single-tile
blocks still need explicit graph lowering rules before generation.

The `examples/conv_flatten_dense_onnx_generate.py` script builds a compact
performance-check model:

```text
Depthwise 3x3 over 6 channels -> ReLU -> 1x1 pointwise Conv ->
Flatten -> Dense -> ReLU -> Dense logits
```

It can also append ONNX Softmax with `--include-softmax` for semantic
inspection. GAIA-ML recognizes that Softmax but rejects project generation until
the backend has a whole-vector normalization kernel.
