Title: ScoreStop: A Gradient-Based Early Stopping Mechanism via Functional Score Tests

Abstract: To prevent overfitting, gradient boosted decision trees necessitate a robust stopping criterion. Conventional approaches rely on monitoring validation loss, halting training if performance plateaus for a predetermined number of iterations. However, this method suffers from the lack of an interpretable scale for the patience parameter and is susceptible to noise in validation losses, which are sometimes implicitly defined by user-specified gradients. In response, we introduce ScoreStop, an early-stopping framework that evaluates the stopping decision at each iteration by testing the null hypothesis that the current predictor minimizes population risk. This approach employs a functional score test calculated on validation data. The resulting statistic is scale-invariant with respect to the update direction and follows a known asymptotic distribution under the null hypothesis. By leveraging gradients instead of loss values, ScoreStop is adaptable to implicit losses like LambdaRank and data-dependent losses such as Cox regression, the latter handled through influence functions. Empirical evaluations on both synthetic datasets and real-world benchmarks demonstrate that ScoreStop performs competitively against traditional loss-based methods.

Source: arXiv Generated at: 2026-06-03 00:00:00 UTC