Title: TabPrep: Bridging the Feature Engineering Void in Tabular Benchmarks

Abstract: While advancements in tabular machine learning have predominantly centered on the development of increasingly complex model architectures, feature engineering remains a pivotal but frequently overlooked element of practical modeling workflows. This critical component is notably missing from contemporary benchmarks, resulting in an unmeasured disparity in evaluation standards. To address this, we present TabPrep, a streamlined preprocessing framework built around feature generators specifically engineered to identify three distinct structural data patterns. Our analysis reveals that numerous common model classes possess predictable vulnerabilities to these patterns, demonstrating that systematic feature engineering can independently drive new performance peaks. When applied to the TabArena benchmark, the integration of TabPrep into model training and tuning processes yields consistent performance enhancements across tree-based, neural, linear, and foundation models, frequently outperforming improvements derived solely from model-centric innovations. Furthermore, TabPrep surpasses prior automated feature engineering methods in terms of performance, efficiency, and cross-dataset applicability, facilitating its adoption in large-scale evaluations. By open-sourcing TabPrep (available at https://github.com/atschalz/tabprep), we provide the research community with the tools to incorporate feature engineering into their benchmarking protocols, thereby rectifying a long-standing deficiency in tabular assessment methodologies.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC