nunchaku-rocm
Copyright 2024 MIT HAN Lab
Copyright 2026 contributors to nunchaku-rocm

This product is a derivative work of Nunchaku
(https://github.com/nunchaku-ai/nunchaku), Copyright 2024 MIT HAN Lab,
licensed under the Apache License, Version 2.0. See LICENCE.txt for the
full license text.

Modifications from the upstream project, made to support AMD ROCm/Triton
GPUs (which have no NVIDIA-only mma.m16n8k32-based CUDA extension):

  - A new portable Triton kernel backend under nunchaku/ops/triton/,
    reimplementing the CUDA kernels in src/kernels/ (W4A4/W8A8 GEMM with
    fused RMSNorm+RoPE and next-layer-quantization epilogues, AWQ, fp16
    GEMM, attention, layernorm, activation, and misc kernels), plus a
    checkpoint-layout inverter that converts existing SVDQuant checkpoints
    from the CUDA kernel's mma-fragment layout to natural row-major layout.
  - Lazy backend selection (nunchaku/ops/backend.py, nunchaku/ops/dispatch.py)
    so `import nunchaku` and model construction succeed without the compiled
    CUDA extension, selectable via the NUNCHAKU_BACKEND environment variable.
  - Corresponding model-level wiring and tests under tests/triton/.
  - FLUX.2 support (nunchaku/models/transformers/transformer_flux2.py),
    adapted from the community vitoom-nunchaku fork of Nunchaku
    (https://huggingface.co/tonera/vitoom-nunchaku, Apache License 2.0),
    with its CUDA-only packed-QKV attention path replaced by the Triton
    attention kernel.
  - A hand-written HIP int4 W4A4 GEMM for AMD RDNA3 (gfx11) GPUs
    (nunchaku/ops/triton/native/), using the v_wmma_i32_16x16x16_iu4
    instruction that Triton cannot emit, compiled at runtime with hipRTC
    and launched through the HIP runtime via ctypes; the Triton kernel
    remains the fallback and the path on every other GPU.
  - ComfyUI custom nodes (comfyui/) that load Nunchaku FLUX.1, FLUX.2,
    Qwen-Image, Z-Image and SDXL checkpoints as ComfyUI models through thin
    adapters on ComfyUI's own model code, including per-block CPU offload for
    Qwen-Image (ComfyUI itself is not included).
  - Runtime LoRA support (nunchaku/lora/runtime.py) that appends LoRAs to
    the SVDQuant layers' low-rank branch.
  - Triton-backend loading for the Qwen-Image, Z-Image and SDXL models
    (checkpoint layout conversion, Z-Image's rotary-embedding layout and
    fused SwiGLU MLP) and fixes for diffusers 0.41's Qwen-Image RoPE and
    Z-Image forward signatures.

The CUDA extension and everything that only worked through it were removed
(this is a real feature loss, not a refactor):

  - The CUDA build machinery: src/ (kernel sources), nunchaku/csrc/
    (pybind/C++ headers), third_party/ (cutlass, Block-Sparse-Attention,
    json, mio, spdlog), setup.py, and the wheel-building CI workflows and
    scripts/ that depended on them.
  - The legacy, CUDA-only model classes NunchakuFluxTransformer2dModel and
    NunchakuSanaTransformer2DModel, and everything that only worked through
    them: TeaCache, PuLID, IP-Adapter integration, the legacy
    caching/diffusers_adapters Flux and SANA adapters, and all top-level
    examples/*.py, tests/flux/, tests/sana/, and the corresponding demo
    apps under app/.
  - Documentation describing the removed features (docs/source/usage/lora.rst,
    attention.rst, qencoder.rst, offload.rst, the PuLID/IP-Adapter usage
    pages, and the CUDA/Windows build guide) and the now-unused Doxygen
    config for the deleted C++ sources.

SANA, TeaCache, PuLID, and IP-Adapter have no equivalent here. See the
commit history for the itemized list of what was removed and why.

README.md was rewritten for this project; the upstream README is kept
unmodified (apart from a note at the top) as README_upstream.md.
