Title: Masked Attention Alignment for Data-Free Quantization of Vision Transformers: Selective Coupling of Decoupled Informative Regions

Abstract

Data-Free Quantization (DFQ) mitigates data privacy risks by generating synthetic samples, thereby eliminating the need for access to original datasets. This technique has attracted significant interest within the domain of Vision Transformers (ViTs), leveraging the advantages of self-attention mechanisms over traditional convolutional operations. Nevertheless, existing DFQ methods for ViTs frequently encounter performance limitations due to distribution discrepancies between the generated synthetic data and the input distribution required by the quantized models (Q).

In this study, we introduce MaskAQ, a novel Masked Attention Alignment framework designed for the data-free quantization of ViTs. Our approach is grounded in two key insights: first, that semantic information within the self-attention mechanism is primarily concentrated in a sparse subset of patches, referred to as informative regions; and second, that these informative regions are the primary drivers of mutual information between synthetic samples and the outputs of Q. To address this, we employ differential entropy maximization based on patch similarity to separate informative regions from noisy backgrounds.

To ensure compatibility with diverse quantized models, we implement a masked attention alignment objective that selects informative regions to synchronize full-precision models with Q, thereby producing high-fidelity synthetic samples. Additionally, we introduce a periodic sample refreshing mechanism, enabling MaskAQ to continuously adapt to the evolving state of Q during training and maintain robust mutual information with the synthetic data. Comprehensive experiments demonstrate that MaskAQ outperforms current state-of-the-art methods across various backbones and downstream tasks. The source code is publicly accessible at https://github.com/hfutqian/MaskAQ.

Source: arXiv Generated at: 2026-06-04 00:00:00 UTC