Title: Enhancing Zero-Shot Audio-Language Classification with Text-Derived Noise Robustness via Drift-Augmented Scoring

Abstract:

Contrastive audio-language models, exemplified by CLAP, facilitate zero-shot audio classification by assigning labels to sounds based on the similarity between audio embeddings and text prompt embeddings, eliminating the need for labeled audio data. However, this matching mechanism is highly susceptible to acoustic noise, leading to significant performance drops. On standard benchmarks, accuracy and mean Average Precision (mAP) decrease by 12 to 30 percentage points when the signal-to-noise ratio (SNR) reaches 0 dB.

To address this vulnerability, we introduce Drift Augmented Scoring (DAS). This method incorporates a minor per-class bonus into the cosine similarity score. The bonus is awarded when the embedding of noisy audio shifts in the direction anticipated by the class’s noise-conditioned text prompts. Because this bonus is derived exclusively from text data, it can be calculated once and cached, requiring only a single inner product computation per class during inference. Notably, DAS operates without gradients or the need for test-time batching.

We evaluated DAS using a LAION CLAP backbone, comparing it against four variants of the concurrent method proposed by Acevedo et al. The evaluation utilized the UrbanSound8K dataset and the complete FSD50K evaluation set, introducing urban acoustic scene noise to audio clips across various SNR levels. DAS demonstrated consistent improvements across all test conditions, boosting accuracy by 2.60 to 5.75 points on UrbanSound8K and increasing mAP by 1.50 to 1.74 points on FSD50K.

Source: arXiv Generated at: 2026-06-04 00:00:00 UTC