SimpleTuner CuTe attention

Kernel implementation synchronized with KohakuFA-CuTe 0.4.0 revision
c373d7bf0a7096a9be0fcc7e488191414aac959c.
https://github.com/SimpleTuner-io/KohakuFA-CuTe

The implementation is distributed under AGPL-3.0-or-later;
see LICENSE.AGPL-3.0. SimpleTuner retains its training backend policy,
package namespaces and public attention adapters.

Explicit 2D neighborhood attention supports shifted boundary windows without
a NATTEN dependency. It uses normalized dQ and reconstructed dK corrections
by default, with bounded H100 scheduling and owned-gradient spatial traversal.

Supported CuTe architectures are SM80, SM86, SM89, SM90, SM100, SM103,
SM120 and SM121. The default arithmetic preserves unscaled score maxima,
separate logarithmic normalization, one input-dtype conversion of P and dS,
FP32 product accumulation and late dQ/dK scaling. Backward uses the actual
rounded forward output for its output/dO correction. Compensated precision
helpers remain explicit experiments, outside default dispatch.

The optional FA4-derived schedules retain their pinned runtime and source
eligibility checks. Atomic backward follows the live deterministic policy;
owned-gradient kernels retain their independent reduction semantics.
Native caches include the C++ source hash, and logical negative/conjugate
view values are resolved before the DLPack boundary.

The producer/consumer architecture in blackwell_pipeline.py adapts KohakuFA's
sm100/fwd.py and sm100/bwd.py at revision
0c5ede6859307bedc0c6089523c16b1ca51b7da9:
https://github.com/KohakuBlueleaf/KohakuFA
KohakuFA is distributed under Apache-2.0, reproduced in LICENSE.Apache-2.0.
The CuTe port retains centered backward and single-owner gradient tiles.

row_view and operand_a adapt accumulator-layout transforms from NVIDIA CUTLASS:
examples/python/CuTeDSL/cute/ampere/kernel/attention/flash_attention_v2.py
WGMMA and TMA API usage follows the BSD-licensed Hopper examples:
examples/python/CuTeDSL/cute/hopper/kernel/dense_gemm/dense_gemm.py
hopper_operand adapts the register-operand layout conversion from:
examples/python/CuTeDSL/cute/hopper/kernel/attention/fmha.py
https://github.com/NVIDIA/cutlass
Reference revision: 0b55a2f691d69981583568fd9eb69687b1f0de8a
Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES.
BSD-3-Clause; the full license is included in LICENSE.

Packed Float32-to-BFloat16/Float16 conversion adapts the cvt_f16x2_f32 and
cvt_f16 helpers from FlashAttention:
flash_attn/cute/utils.py
https://github.com/Dao-AILab/flash-attention
Reference revision: 85d6dfe3573708b5632915950cd8b069b924ba52
Copyright (c) 2025, Tri Dao.
BSD-3-Clause; the full license and contributor list are included in LICENSE.

The optional fa4_sm80 package derives modified forward, softmax, backward, and
preprocess methods from FlashAttention at revision
4616f5116d33e3ba3cbb71d582caffcfcb49649c. Unchanged methods use the pinned optional
dependency. Copyright and contributor notices are retained with the derived
methods; the BSD-3-Clause license and AUTHORS list are reproduced in
LICENSES/FlashAttention-BSD-3-Clause.txt and LICENSES/FlashAttention-AUTHORS.txt.

The experimental fa4_hopper package derives modified Hopper WGMMA forward,
softmax, backward, preprocessing and persistent scheduling methods from the
same FlashAttention revision. It preserves separate unscaled maxima and
logarithmic normalization, actual rounded-output delta, input-dtype P before
dS, one dS conversion, FP32 accumulation and late dQ/dK scaling. Unchanged helpers
use the pinned optional dependency. The derived methods retain copyright and
contributor notices under the same reproduced BSD-3-Clause license.
