Title: Channel-wise Vector Quantization

Abstract: This paper introduces Channel-wise Vector Quantization (CVQ), an innovative image tokenization approach that substitutes traditional patch-wise tokens with channel-wise tokens. While standard vector quantization maps discrete tokens to individual patch feature vectors, CVQ quantizes the channels of the feature map instead. This method characterizes an image through discrete levels of visual detail rather than as a spatial grid of patches. Leveraging CVQ, we propose a new visual autoregressive framework centered on "next-channel prediction." Our Channel-wise Autoregressive (CAR) model generates images by predicting channels in sequence, thereby creating progressively richer visual details. Rather than assembling images patch by patch in raster order, CAR first outlines the global structure and subsequently refines fine-grained attributes, mimicking the process of a human artist. Our empirical results demonstrate two key advantages: (1) CVQ secures 100% codebook utilization with a codebook size exceeding 16K without additional optimizations, while significantly enhancing reconstruction quality compared to conventional VQ; and (2) CAR achieves a DPG score of 86.7 and a GenEval score of 0.79, highlighting its robust performance in text-to-image generation tasks.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC