Title: PathAR: A Structure-First Autoregressive Approach to Multimodal Pathology Image Synthesis

Abstract:

The challenge of limited data availability in multimodal pathology drives the development of unified generative models capable of synthesizing modality-specific visual characteristics while maintaining anatomically coherent structural integrity. While different imaging modalities exhibit distinct appearance statistics, core morphological features—such as tissue boundaries and cellular topology—remain largely consistent across various acquisition protocols. However, current methodologies frequently treat these elements within a uniform token stream, which implicitly entangles structure with appearance and reduces structural controllability when modalities change.

To overcome these limitations, we introduce PathAR (Pathology Autoregressive modeling), a novel framework for structure-first autoregressive synthesis designed for modality-label-conditioned pathology generation. This approach explicitly separates structure and appearance. PathAR utilizes a dual vector quantization (Dual-VQ) tokenizer to decompose input samples into tokens representing mask-grounded structure and tokens representing appearance. Furthermore, it employs an interleaved autoregressive (IAR) transformer featuring asymmetric attention visibility to strictly enforce the dependency of appearance on structure.

This design ensures that morphology remains stable despite heterogeneous, modality-specific appearances and facilitates the generation of spatially aligned image-mask pairs. Comprehensive experiments demonstrate that PathAR outperforms baseline methods in both structural consistency and modality fidelity. Additionally, the model preserves sample diversity, supports downstream segmentation tasks in data-scarce environments, and shows potential for extending to finer-grained variations within specific organ labels across modalities.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC