Title: Utilizing Embedding-Space Diffusion for Zero-Shot Environmental Sound Classification

Abstract:

Zero-shot learning allows models to generalize to previously unseen classes by utilizing semantic information, thereby bridging the divide between training and testing sets that contain non-overlapping categories. Although significant research efforts have been directed toward zero-shot learning in computer vision, its application to environmental audio has received limited attention, resulting in suboptimal performance in current studies. Notably, while generative approaches have proven successful in visual domains, they are largely absent from the field of zero-shot environmental sound classification. To fill this void, this study explores the use of generative methods for zero-shot learning within environmental audio contexts. We adapt two prominent generative models from computer vision: the cross-aligned and distribution-aligned variational autoencoder (CADA-VAE) and the leveraging invariant side generative adversarial network (LisGAN). Furthermore, we propose a novel diffusion model conditioned on auxiliary class data. In this framework, synthetic embeddings produced by the diffusion model are merged with embeddings from seen classes to train a classifier. Our experiments were carried out across five environmental audio datasets—ESC-50, ARCA23K-FSD, FSC22, UrbanSound8k, and TAU Urban Acoustics 2019—as well as one music classification dataset, GTZAN. The findings indicate that the diffusion model surpasses all baseline methods on average across the six audio datasets evaluated. This research positions the diffusion model as a highly promising strategy for zero-shot learning and presents the first benchmark for generative methods in zero-shot environmental sound classification, laying the groundwork for subsequent investigations.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC