Metadata-Version: 2.4
Name: xtc-tools
Version: 0.3.1
Author-email: Hugo Pompougnac <hugo.pompougnac@inria.fr>
License-Expression: BSD-3-Clause
Platform: Linux
Classifier: Programming Language :: Python :: 3
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: LICENSE_HEADER
Requires-Dist: jinja2
Requires-Dist: matplotlib
Requires-Dist: numpy<2.3
Requires-Dist: ordered-set
Requires-Dist: py-cpuinfo
Requires-Dist: tqdm
Requires-Dist: typing_extensions
Requires-Dist: pyyaml
Requires-Dist: scikit-learn
Requires-Dist: xdsl~=0.57.1
Requires-Dist: strictyaml
Requires-Dist: tomlkit
Provides-Extra: default
Requires-Dist: xtc-llvm-tools==22.1.8.2; extra == "default"
Requires-Dist: xtc-mlir-tools==22.1.8.3; extra == "default"
Requires-Dist: xtc-mlir-python-bindings==22.1.8.3; extra == "default"
Requires-Dist: xtc-mlir-extra-tools==22.1.8.7; extra == "default"
Requires-Dist: apache-tvm==0.26.0; extra == "default"
Requires-Dist: apache-tvm-ffi==0.1.13.post3; extra == "default"
Provides-Extra: dev
Requires-Dist: xtc-llvm-tools==22.1.8.2; extra == "dev"
Requires-Dist: xtc-mlir-tools==22.1.8.3; extra == "dev"
Requires-Dist: xtc-mlir-python-bindings==22.1.8.3; extra == "dev"
Requires-Dist: xtc-mlir-extra-tools==22.1.8.7; extra == "dev"
Requires-Dist: apache-tvm==0.26.0; extra == "dev"
Requires-Dist: apache-tvm-ffi==0.1.13.post3; extra == "dev"
Requires-Dist: coverage>=7.8.0; extra == "dev"
Requires-Dist: filecheck==1.0.3; extra == "dev"
Requires-Dist: lit; extra == "dev"
Requires-Dist: mkdocs; extra == "dev"
Requires-Dist: mkdocs-material; extra == "dev"
Requires-Dist: mkdocstrings-python; extra == "dev"
Requires-Dist: mypy==1.15.0; extra == "dev"
Requires-Dist: pytest; extra == "dev"
Requires-Dist: pytest-xdist; extra == "dev"
Requires-Dist: pyright==1.1.407; extra == "dev"
Requires-Dist: types-PyYAML; extra == "dev"
Requires-Dist: marimo==0.19.6; extra == "dev"
Requires-Dist: ruff==0.14.10; extra == "dev"
Provides-Extra: test
Requires-Dist: coverage>=7.8.0; extra == "test"
Requires-Dist: filecheck==1.0.3; extra == "test"
Requires-Dist: lit; extra == "test"
Requires-Dist: mkdocs; extra == "test"
Requires-Dist: mkdocs-material; extra == "test"
Requires-Dist: mkdocstrings-python; extra == "test"
Requires-Dist: mypy==1.15.0; extra == "test"
Requires-Dist: pytest; extra == "test"
Requires-Dist: pytest-xdist; extra == "test"
Requires-Dist: pyright==1.1.407; extra == "test"
Requires-Dist: types-PyYAML; extra == "test"
Requires-Dist: marimo==0.19.6; extra == "test"
Requires-Dist: ruff==0.14.10; extra == "test"
Provides-Extra: mlir
Requires-Dist: xtc-llvm-tools==22.1.8.2; extra == "mlir"
Requires-Dist: xtc-mlir-tools==22.1.8.3; extra == "mlir"
Requires-Dist: xtc-mlir-python-bindings==22.1.8.3; extra == "mlir"
Requires-Dist: xtc-mlir-extra-tools==22.1.8.7; extra == "mlir"
Provides-Extra: tvm
Requires-Dist: apache-tvm==0.26.0; extra == "tvm"
Requires-Dist: apache-tvm-ffi==0.1.13.post3; extra == "tvm"
Dynamic: license-file

![logo.png](logo.png)

[![codecov](https://codecov.io/github/xtc-tools/xtc/graph/badge.svg)](https://codecov.io/github/xtc-tools/xtc)

# XTC

## Links

Refer to documentation at https://xtc-tools.github.io/xtc

Refer to tutorials [here](docs/tutorials) and to additionnal developers documentation [here](docs/develop).

## Overview

XTC is a domain-specific dataflow graph compiler for linear algebra operations. It provides:
- **Operational DSL**: Define computation graphs with tensors and operators
- **Scheduling DSL**: High-level transformations (tiling, parallelization, vectorization, etc.)
- **Multiple backends**: MLIR (linalg + transform), TVM (Tensor IR), JIR (INRIA internal)
- **Autotuning**: Definition and exploration of the optimization space

## Build & Development

### Installation

If needed, install uv following the instructions [here](https://docs.astral.sh/uv/).

Debian-like x86_64 or aarch64 Linux distributions (Python: 3.10 to 3.14 inclusive):
```bash
sudo apt install python3 python3-dev build-essential libomp5 binutils binutils-aarch64-linux-gnu binutils-x86-64-linux-gnu
sudo apt install libpfm4-dev # Optional: interface to Linux perf counters
sudo sysctl kernel.perf_event_paranoid=1 # Optional: give access to hardware counters
uv venv -p 3.12 && source .venv/bin/activate
uv pip install -e '.[dev]'
make test
```

MacOs M1+ macos-14/macos-15 (Python: 3.10 to 3.14 inclusive):
```bash
brew install libomp x86_64-linux-gnu-binutils aarch64-elf-binutils
export DYLD_LIBRARY_PATH="/opt/homebrew/opt/libomp/lib:$DYLD_LIBRARY_PATH"
uv venv -p 3.12 && source .venv/bin/activate
uv pip install -e '.[dev]'
make test
```

Note that `[dev]` extension installs both `mlir` and `tvm` backends in addition to development tools.

Available extensions are:
- `[mlir]`: MLIR backend
- `[tvm]`: TVM backend
- `[test]`: develop, docs and tests tools
- `[default]`: mlir + tvm
- `[dev]`: default + test

### Using a local LLVM / MLIR build

By default XTC uses the LLVM/MLIR toolchain shipped in its Python wheels. To
use a local LLVM checkout instead (e.g. to test compiler changes), point XTC
at your build directory (`$LLVM_BUILD` below). Two levels are possible.

**Binaries only.** Override `opt`, `llc`, `mlir-opt` and `mlir-translate`
while keeping the wheel's Python bindings — the simplest option, e.g. when
your changes live in the LLVM middle-end / back-end:

```bash
export XTC_LLVM_PREFIX=$LLVM_BUILD   # providing bin/opt and bin/llc
export XTC_MLIR_PREFIX=$LLVM_BUILD   # providing bin/mlir-opt and bin/mlir-translate
```

**Binaries and Python bindings.** The MLIR bindings are a native extension,
so the interpreter must match the Python version they were built for (check
the ABI tag under
`$LLVM_BUILD/tools/mlir/python_packages/mlir_core/mlir/_mlir_libs`).
Create a matching venv if, install XTC without the MLIR wheels, build the
runtime support libraries, then prepend the bindings to `PYTHONPATH`:

```bash
uv venv -p 3.14 .venv-local && source .venv-local/bin/activate
uv pip install -e '.[tvm]'

export PYTHONPATH=$LLVM_BUILD/tools/mlir/python_packages/mlir_core:$PYTHONPATH
export XTC_MLIR_PREFIX=$LLVM_BUILD
export XTC_LLVM_PREFIX=$LLVM_BUILD
export XTC_MLIR_TARGET=llvmir
```

### Code quality

Code quality requirements:
- **Type annotations**: Strict pyright mode, full annotations required
- **Formatting**: Ruff (line length 88)
- **License headers**: BSD-3-Clause required on all source files
- All checks must pass before merge

Type checking:
```bash
make check-type          # Run both pyright and mypy
pyright                  # Run pyright only
mypy                     # Run mypy only
```

Formatting:
```bash
make format              # Apply all formatting (license + ruff)
make check-format        # Check formatting without modifying files
```

Testing structure:
- `tests/pytest/unit/`: Core interface unit tests
- `tests/pytest/{mlir,tvm}/`: Backend-specific tests
- `tests/filecheck/`: Lit+FileCheck functional tests for code generation

Global test commands:
```bash
make test                # Run minimal unit tests
make check               # Run ALL acceptance tests (required for contributions)
make check-pytest        # Run pytest suite only
make check-lit           # Run LIT tests for LLVM IR target
make check-lit-c         # Run LIT tests for C target
pytest tests/pytest/unit # Run specific test directory
```

Running individual tests:
```bash
# Single pytest file
pytest tests/pytest/unit/test_specific.py -v

# Single lit test
lit -v tests/filecheck/backends/specific_test.py

# C target for lit tests
XTC_MLIR_TARGET=c lit -v tests/filecheck/backends/specific_test.py
```

### Dependencies update

Python package dependencies are listed in `dependencies.toml` with definition of groups and groups
dependencies.

Always update dependencies there and run `make dependencies` to update `pyproject.toml` before commit.

### Releases

Package versions are derived from Git tags. Successful updates to `main` are
published to TestPyPI as development versions, while tags of the form
`xtc-vX.Y.Z` are published to PyPI. See
[the release guide](docs/develop/releasing.md) for the release procedure and
required trusted-publishing configuration.

## Architecture

### Core Abstractions (src/xtc/itf/)

Abstract interfaces defining the compilation pipeline:
- `data/` - Tensor, DataType, ShapeType
- `operator/` - Linear algebra operator interface
- `graph/` - Graph, Node, Operation abstractions
- `back/` - Backend interface
- `schd/` - Scheduler and Schedule abstractions
- `comp/` - Compiler interface
- `exec/` - Executor and Evaluator interfaces
- `search/` - Search space exploration interface

### Backends (src/xtc/backends/)

Exposed backends:
- `mlir/` - MLIR backend using linalg + transform dialects
- `tvm/` - TVM backend using Tensor IR + Schedule APIs

XTC also supports multiple MLIR Targets for the code generation:
  - llvmir (default)
  - c
  - nvgpu

To force the use of a specific target, you can set the env variable `XTC_MLIR_TARGET=<mlir-target>`.

The MLIR backend can be extended using the SDist extension, which provides distribution primitives.
To install SDist, follow the instructions in 
[docs/develop/optional_backends.md](docs/develop/optional_backends.md), in the "MLIR development version" section.

Note that the nvgpu target requires a recent version of Cuda (tested with Cuda 13.0).
By default, it tries to find Cuda at */usr/local/cuda*, but it can be overridden with the CUDA_INSTALL_DIR env variable.
The performance counters can be accessed is the GPU has a compute capability >=7.5.

### Compilation Pipeline

1. User defines Graph with Tensors and Operators
2. Backend created from Graph
3. Scheduler applies transformations and produces Schedule
4. Compiler generates executable Module
5. Executor/Evaluator runs and measures performance

### CLI Tools (src/xtc/cli/)

- `mlir-loop` - High-level scheduling for MLIR linalg operators
- `mlir-backend` - MLIR backend wrapper
- `loop-explore` - Autotuning and space exploration
- `loop-display` - Visualization of exploration results

## AI assistants

To create agent guidance files from this README: `make agents` (AGENTS.md) or `make claude` (CLAUDE.md)
