Title: Revisiting Tree Count Optimization in Random Forests: Integrating Plateau Search with Optuna

Abstract

Optimizing the hyperparameters of Random Forests presents a unique challenge when determining the optimal number of trees. Because predictive performance generally improves monotonically as the ensemble grows, conventional optimization techniques like Tree-structured Parzen Estimator (TPE) and Hyperband necessitate a fixed search interval. These methods frequently push the estimated value to the upper limit of that range. While early-stopping mechanisms eliminate the need for predefined bounds, they are often vulnerable to score fluctuations and may halt the process too early.

To overcome these limitations, we introduce a novel algorithm that integrates a triplet-based plateau search with Optuna. This approach excludes the tree count from the direct TPE search space while still leveraging insights gained from previous hyperparameter optimization (HPO) trials. By observing relative shifts in out-of-bag (OOB) scores across a sliding window of three distinct forest sizes, the method dynamically tracks the smallest ensemble size that remains sufficient for performance. This window is adjusted automatically, creating a transparent procedure governed by a single tolerance parameter.

We also present a theoretical framework linking our relative OOB-score criterion to the divergence between current and asymptotic performance limits, along with an asymptotic variance estimate for the absolute relative difference in OOB scores. Empirical results indicate that the optimal tree count often deviates significantly from standard heuristics. On most traditional benchmark datasets, the required number of trees is lower, whereas high-dimensional bioinformatics datasets, specifically Arcene and Dorothea, benefit from larger ensembles. The source code and reproducible experimental data are hosted at https://github.com/lange-am/rf_plateau_hpo.

Source: arXiv Generated at: 2026-06-03 00:00:00 UTC