Title: CRePE: Convolution-aware Relative Importance in Post-training Pruning with Efficient Search

Abstract

The practical deployment of Large Language Models (LLMs) is often hindered by significant memory and computational demands. Post-training pruning (PTP) offers a viable solution to mitigate these expenses by eliminating weights without the need for additional training. While the RIA method has achieved state-of-the-art accuracy by utilizing relative importance scores normalized via row and column sums, it is limited to 1D cross-shaped (row/column) directional data and treats row and column contributions with equal weight. To overcome these limitations, we introduce CRePE, a novel approach that enhances Relative Importance scoring by integrating 2D local neighborhood contexts and adaptive coefficients. CRePE demonstrates superior performance compared to current PTP techniques across various models and sparsity levels.

However, determining the optimal adaptive coefficients through perplexity (PPL)-based hill climbing is resource-intensive, requiring extensive PPL evaluations and roughly 11 hours of search time. We address this bottleneck by proposing PHO (Proxy-based Hyperparameter Optimization), a method that removes the necessity for repeated PPL measurements and shrinks the search duration to approximately 20 minutes. Additionally, the hyperparameter configurations identified by PHO exhibit strong generalization capabilities, transferring effectively to other models. Finally, we confirm that CRePE can be seamlessly integrated with existing techniques, such as Channel Permutation, non-uniform sparsity allocation, and re-pruning methods.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC