Title: LeAP: Learnable Adaptive Permutation for Feature Selection in Heterogeneous and Sparse Recommender Systems

Abstract

Contemporary industrial recommender systems depend on a vast array of heterogeneous features to deliver high-precision predictions. These features span a wide spectrum, including low-dimensional scalars, such as statistical values, and high-dimensional embeddings, like user-ID vectors and MLP representations. Given the substantial computational burden associated with model training, efficient feature selection is paramount. However, current methodologies face three significant limitations: first, they generally presume uniform feature dimensions or necessitate expensive mappings to a fixed size; second, they are ill-equipped to handle extreme sparsity, where over 99% of features often retain default values; and third, conventional permutation-based techniques are too computationally expensive for large-scale deployment.

To overcome these obstacles, we introduce LeAP (Learnable Adaptive Permutation), a novel, model-agnostic plug-in module designed for feature selection. LeAP converts the inefficient random permutation process into a learnable mechanism, thereby drastically speeding up the assessment of feature importance. Additionally, we present an adaptive regularization strategy specifically engineered for heterogeneous dimensions and high levels of sparsity. This approach facilitates superior ranking of feature importance across asymmetric input spaces.

Evaluations on four public recommendation datasets confirm that LeAP delivers state-of-the-art performance. Moreover, the module has been successfully integrated into a large-scale industrial search ranking model handling more than one billion daily requests and comprising a 2TB parameter scale. In this real-world environment, which involves over 12,000 total feature dimensions, LeAP effectively identified and eliminated more than 3,600 redundant dimensions without any loss in performance. This capability represents a 2 to 10-fold improvement over comparable baseline methods.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC