FlashRWKV2

This repository began from MoonshotAI/FlashKDA, distributed under the MIT
License. The original KDA attention operator, Python package, tests, and
benchmark data-plane were removed. The following new RWKV-7 files retain the
MIT boundary and derive their explicit K1/K2 launch separation from
MoonshotAI/FlashKDA commit
1ce47ea3bb22c84eb9cc665028399cf35e8ffb0b:

- csrc/infer/wkv7/infer_common_chunk_bf16_forward_k1_prepare.cu
- csrc/infer/wkv7/infer_common_chunk_bf16_forward_k2_recurrence.cu
- csrc/infer/wkv7/infer_common_chunk_bf16_bindings.cpp

They implement the canonical RWKV-7 recurrence, not the KDA attention
mathematics.

The following files contain material adapted from the vLLM project and its
RWKV/Albatross integration at rwkv-rs/vllm-rwkv commit
6d683f9e49a2997e405c47edc147872c8609513b:

- csrc/infer/wkv7/infer_common_recurrent_varlen_bindings.cpp
- csrc/infer/wkv7/infer_common_recurrent_fp32io16_forward_varlen.cu
- csrc/infer/wkv7/infer_sm80_recurrent_fp16_forward_varlen.cu
- benchmarks/infer/wkv7/benchmark_infer_recurrent_fp16_fp32io16_forward_varlen.py
- benchmarks/benchmark_rwkv7.py
- benchmarks/benchmark_training.py
- benchmarks/kernel_benchmark.py

The FP16 recurrent lineage includes BlinkDL/Albatross faster3a_2607 commit
63c53f4abf2cd891dd3a18c8f44f5b2cccc8c64b.

The following fused FP16 inference block operators are adapted from
BlinkDL/Albatross faster3a_2607 commit
ee3308f6922e59f2166c7fac3c5a192340a2b48e:

- csrc/infer/tmix/infer_common_tmix_fp16_forward_registration.cpp
- csrc/infer/tmix/infer_common_tmix_fp16_forward.cu
- csrc/infer/cmix/infer_common_cmix_fp16_forward_registration.cpp
- csrc/infer/cmix/infer_common_cmix_fp16_forward.cu
- csrc/infer/elementwise/infer_common_elementwise_fp16_forward_registration.cpp
- csrc/infer/elementwise/infer_common_elementwise_fp16_forward.cu

The already-owned FP16/FP32 WKV recurrence files from that source are not
duplicated. These derived files are licensed under the Apache License,
Version 2.0; see LICENSES/Apache-2.0.txt.

The following files contain recurrent checkpoint and reverse-recurrence
methods adapted from BlinkDL/RWKV-LM commit
952102498e9ed367ea0a59ee64106916d474d30f:

- csrc/common/wkv7/recurrent_common_fp32io16.cpp
- csrc/common/wkv7/recurrent_common_fp32io16.h
- csrc/common/wkv7/recurrent_common_fp32io16_forward.cu
- csrc/common/wkv7/recurrent_common_fp32io16_backward.cu
- csrc/pretrain/wkv7/pretrain_common_recurrent_fp32io16_bindings.cpp
- csrc/statetune/wkv7/statetune_common_recurrent_fp32io16_bindings.cpp
- csrc/pretrain/wkv7/pretrain_common_head_l2wrap_ce_bf16_forward_registration.cpp
- csrc/pretrain/wkv7/pretrain_common_head_l2wrap_ce_bf16_forward.cu
- csrc/pretrain/wkv7/pretrain_common_cmix_bf16_forward_backward_registration.cpp
- csrc/pretrain/wkv7/pretrain_sm90_cmix_bf16_forward_backward.cu
- csrc/pretrain/wkv7/pretrain_common_l2wrap_ce_bf16_forward_backward_registration.cpp
- csrc/pretrain/wkv7/pretrain_common_l2wrap_ce_bf16_forward_backward.cu
- csrc/pretrain/wkv7/pretrain_common_tmix_a_gate_bf16_forward_backward_registration.cpp
- csrc/pretrain/wkv7/pretrain_common_tmix_a_gate_bf16_forward_backward.cu
- csrc/pretrain/wkv7/pretrain_common_tmix_kk_pre_bf16_forward_backward_registration.cpp
- csrc/pretrain/wkv7/pretrain_sm90_tmix_kk_pre_bf16_forward_backward.cu
- csrc/pretrain/wkv7/pretrain_common_tmix_lnx_rkvres_xg_bf16_forward_backward_registration.cpp
- csrc/pretrain/wkv7/pretrain_common_tmix_lnx_rkvres_xg_bf16_forward_backward.cu
- csrc/pretrain/wkv7/pretrain_common_tmix_mix6_bf16_forward_backward_registration.cpp
- csrc/pretrain/wkv7/pretrain_sm90_tmix_mix6_bf16_forward_backward.cu
- csrc/pretrain/wkv7/pretrain_common_tmix_vres_gate_bf16_forward_backward_registration.cpp
- csrc/pretrain/wkv7/pretrain_common_tmix_vres_gate_bf16_forward_backward.cu

Those derived files are licensed under the Apache License, Version 2.0 and
retain SPDX/source-revision headers. See LICENSES/Apache-2.0.txt.

The FLA chunk_rwkv7 and fused_recurrent_rwkv7 implementations are used only
through an external correctness and benchmark dependency. No FLA
implementation source is bundled here.

All other original FlashRWKV2 work is licensed under the repository's MIT
license unless a file states otherwise.
