Title: PermuQuant: Reducing Per-Group Quantization Error via Channel Reordering for Diffusion Models

Abstract:

While large-scale visual generative models have demonstrated exceptional performance, their substantial computational and memory demands pose significant barriers to deployment in resource-limited environments, including interactive applications and single-GPU personal setups. Post-training quantization (PTQ) emerges as a viable strategy, enabling the compression of pretrained models without the need for costly retraining. Nevertheless, current PTQ techniques often experience significant quality loss when applied in extreme low-bit regimes. This study highlights channel ordering as a critical yet insufficiently investigated variable within the context of per-group quantization, where each contiguous block utilizes a shared quantization scale. When channels exhibiting disparate statistical properties are grouped together, outlier values can skew the shared scale, resulting in substantial quantization inaccuracies.

Addressing this issue, we introduce PermuQuant, a straightforward and potent PTQ framework tailored for low-bit diffusion models. Prior to per-group quantization, PermuQuant organizes channels according to a joint second-moment metric, thereby clustering channels with analogous activation and weight characteristics. Additionally, the framework employs a calibration-driven acceptance mechanism, implementing reordering only if the chosen permutation demonstrably lowers quantization error on calibration datasets. To maintain efficiency, these permutations are either integrated into neighboring modules or executed on weights offline, eliminating the need for explicit runtime permutation steps. Comprehensive evaluations across various large diffusion models indicate that PermuQuant reliably minimizes quantization error and surpasses existing PTQ baselines. Specifically, when applied to FLUX.1-dev on an RTX 5090 with W4A4 NVFP4 quantization, PermuQuant delivers up to a 1.7-fold speedup per step and cuts the DiT memory usage by a factor of 3.5. The source code will be released at https://github.com/yscheng04/PermuQuant.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC