Title: BAHSD: Addressing the Long-Tail Disparity in Black-Box Sequential Recommendation Through Adaptive Distillation

Abstract:

Although sequential recommendation systems are ubiquitous, they are frequently inaccessible as proprietary black-box APIs. This limitation has spurred growing interest in model extraction techniques designed to replicate their functionality locally. However, the inherent long-tail distribution of user data creates significant signal heterogeneity. Dense head sequences tend to reinforce the teacher’s existing preferences, leading to preference solidification that biases the extraction process toward local patterns. Conversely, sparse tail sequences generate flat, noisy predictions. Current extraction methods that apply a uniform approach fail to account for this disparity, often resulting in noise overfitting and inefficient knowledge transfer.

To address these challenges, we introduce BAHSD, a black-box adaptive distillation framework. BAHSD manages signal heterogeneity by employing a multi-scale consistency probing mechanism, which implicitly assesses signal reliability. Leveraging these insights, we design an adaptive hierarchical objective function. For high-confidence signals, dynamic-temperature KL divergence is utilized to alleviate preference solidification. For low-confidence signals, ranking consistency combined with InfoNCE contrastive learning offers robust enhancement against noise. Experimental results demonstrate that BAHSD consistently surpasses baseline models, delivering up to a 4.98% performance gain over the teacher model and achieving improvements of over 80% for tail users. This framework serves as a plug-and-play solution for extracting high-fidelity recommendations from black-box systems.

Source: arXiv Generated at: 2026-06-03 00:00:00 UTC