Metadata-Version: 2.5
Name: qwen-image-mnn
Version: 0.1.0
Summary: Qwen-Image-2.1 text-to-image on MNN weights (on-device, no torch); fp16 / int8 / int4
Project-URL: Homepage, https://huggingface.co/yunfengwang/Qwen-Image-2.1-MNN-fp16
Author: yunfeng
License: Apache-2.0
Keywords: fp16,int4,int8,mnn,on-device,qwen-image,text-to-image
Requires-Python: >=3.10
Requires-Dist: mnn>=3.6.0
Requires-Dist: numpy>=1.24
Requires-Dist: pillow>=10.0
Requires-Dist: tokenizers>=0.20
Description-Content-Type: text/markdown

# qwen-image-mnn

Run **Qwen-Image-2.1** text-to-image on-device with the **MNN** weights — no torch, no
diffusers, no 33 GB checkpoint. One command through `uvx`:

```bash
uvx qwen-image-mnn "a red panda riding a bicycle" --steps 40 --out panda.png
```

The CLI downloads the converted MNN weights on first use (~30 GB for `--quant fp16`) into
`~/.cache/qwen-image-mnn/<quant>`, then renders offline. The official pipeline defaults are used
unless you say otherwise: 40 steps, 1024 px, `true_cfg_scale=1.0`.

## What runs where

| piece | graph | fp16 | int8 | int4 |
| --- | --- | --- | --- | --- |
| Qwen3-VL text encoder (language model, text-only path) | `text_encoder.mnn` | 15.1 GB | 11.2 GB | 7.7 GB |
| Qwen-Image-2.1 DiT (32 blocks, 4096 dim) | `dit.mnn` | 14.2 GB | 8.9 GB | 5.3 GB |
| VAE (decoder + encoder) | `vae_decoder.mnn`, `vae_encoder.mnn` | 1.3 GB | 1.3 GB | 1.3 GB |
| **download** | | **30.7 GB** | **21.4 GB** | **14.4 GB** |

`int8`/`int4` are weight-only quantizations (`mnnconvert --weightQuantBits … --weightQuantBlock 32
--hqq`) of the same graphs; the VAE stays fp32 at every level. int8 stays inside the model's own
dtype noise — the fp16-vs-bf16 reference run drifts 6.3e-3 → 2.0e-1 while int8 drifts
1.4e-3 → 2.4e-2, image PSNR 52 dB against the reference render (fp16: 43.6 dB). int4 is
measurably worse (4.4e-1, PSNR 28.5 dB): use it when 7 GB matters more than the difference.

Every graph is exported from the released diffusers checkpoint to ONNX and then converted with
`mnnconvert`. The tokenizer, the flow-match scheduler and all the joint-sequence bookkeeping
(3-axis RoPE, block-causal mask, modulation rows) are plain numpy in this package, because they
are cheap — the transformer itself is the only part that needs MNN.

## Install / run

```bash
uvx qwen-image-mnn --help              # from PyPI
uvx --from /path/to/qwen-image-mnn qwen-image-mnn "a photo of a cat" --seed 7   # or a checkout
```

Options that matter:

```
--quant fp16           weight precision to download: fp16 | int8 | int4
--size 1024            square side in pixels (or --height/--width)
--steps 40             denoising steps (40 is the released pipeline's default)
--seed 0               seed for the initial latents
--true-cfg-scale 1.0   >1 with --negative-prompt enables classifier-free guidance
--backend cpu          the only backend the weights were verified on; metal/opencl are
                       experimental on MNN 3.6.1/macOS
--precision high       keep this: `low` swaps the CPU kernels to fp16 *arithmetic*, which
                       overflows the 32-block residual stream (absmax 2.6e36, output full of
                       inf). `high` and `normal` are bit-identical, so there is nothing to gain.
--model-dir DIR        use a local converted-model directory instead of downloading
--latents X.npy        start from fixed packed latents (reproducible comparisons)
--dump-latents X.npy   save every step's latents
--prompt-embeds X.npy  supply prompt embeddings (e.g. saved from the reference pipeline) and
                       skip the text encoder -- this isolates the DiT + VAE when comparing
```

## Speed

CPU only (Metal numbers below). Text-to-image on this 15-core Mac, 10 threads:

| stage | fp16 | int8 | int4 |
| --- | --- | --- | --- |
| text encoder, one prompt | 31 s | 66 s | 20 s |
| DiT forward @ T=1120 (512 px) | 108 s | 42 s | 38 s |
| 8 steps + VAE @ 512 px | 171 s | 277 s | 113 s |
| 40 steps + VAE @ 1024 px (official default) | 80 min | — | — |

The single-forward numbers come from a bandwidth-bound graph: the quantized weights are dequantized
as they page in, so int8/int4 can beat fp16 there even though per-step end-to-end times were
measured under load and vary with what else the machine is doing. Treat the table as ballparks,
not a benchmark.

## How the numbers were verified

Each graph was run against its source with identical inputs:

| graph | vs its ONNX source | vs the released model |
| --- | --- | --- |
| text encoder | 8.4e-4 | 6.0e-3 |
| DiT | 1.7e-2 (max-rel, T=1120) | 3.0e-2 (vs a torch fp16 forward, T=4192) |
| VAE decoder | 1.0e-3 … 4.2e-3 | — |
| VAE encoder | 2.1e-3 … 3.6e-3 | — |

End to end, with both sides starting from the same latents: the MNN pipeline's latents drift
5.0e-3 (step 1) … 3.1e-2 (step 8) relative to the released pipeline's own fp16 run, image
PSNR 43.6 dB. The released model compared against *itself* across dtypes (fp16 vs bf16, same
seed and latents) drifts up to 2.0e-1 in the same job — the conversion is closer to the model
than the model is to itself once the dtype moves. `tools/ref_noise_floor.sh` reproduces that
floor, `tools/compare_e2e.py` is the comparison and its tolerance is sized from it.

The fp32 VAE is what the published numbers were measured with; the fp16 VAE variants kept in
`tools/` are for the record only (the decoder holds PSNR 43 dB, the encoder's mean error grows
to 9e-3).

Text-only conversion notes, the text-encoder bisect (block-level taps that found the MNN
importer bug) and the fixups that rewrite the opset-18 dynamo output into the opset-17 shape
MNN's importer actually reads correctly live in `tools/` (this package's sibling repo):
`export_dit_onnx.py`, `export_text_encoder_onnx.py`, `export_vae_onnx.py`, `onnx_fixups.py`.

## Current scope

Text-to-image (T2I) on the CPU backend. Image editing (the "创改一体" path) needs the vision
tower and the deepstack merge, which is not part of these graphs yet.

Metal on macOS 26 with MNN 3.6.1 miscomputes: the Metal tensor-API capability probe fails to
compile against the new MetalPerformancePrimitives headers and the fallback kernels return
garbage. MNN master fixes the probe, and a locally built master library computes all three graphs
correctly on Metal (VAE max rel 1.5e-3, DiT 2.2e-3) — but only the VAE gets faster (3.1x); the
DiT is ~4x slower than CPU (95.7 s against 24.1 s at T=1120), so CPU is the fastest full path
even once an upstream release ships the fix. The pip wheel cannot load a locally built
`libMNN.dylib` either, so none of this is reachable from this package today.

---

## 中文速览

`uvx qwen-image-mnn "提示词" --steps 40 --out out.png` 即可出图：首次运行按 `--quant` 把权
重下载到 `~/.cache/qwen-image-mnn/<quant>`（fp16 30.7 GB / int8 21.4 GB / int4 14.4 GB），之后
完全离线。默认参数与官方一致：40 步、1024 px。pip 版 MNN 上请用默认 `--backend cpu
--precision high`：`low` 精度会溢出，Metal 在 MNN 3.6.1 结果异常（上游 master 已修探针，但实测
只有 VAE 变快 3.1 倍，DiT 反而比 CPU 慢 4 倍，且 pip 包无法加载自编译库）。int8 与 fp16 同处于
模型自身 dtype 噪声之内，推荐替代；int4 有可测差异（PSNR 28.5 dB vs 52.0 dB），仅在体积敏感时
使用。当前覆盖文生图；图生图/图像编辑还需要视觉塔与 deepstack 融合，尚未包含。
