Beyond Additive Decompositions: Interpretability Through Separability

Abstract

In the realm of interpretable machine learning, achieving a balance between predictive accuracy and structural fidelity to the underlying data is paramount. Current explainability techniques predominantly depend on additive frameworks, such as Generalized Additive Models (GAMs), SHapley Additive exPlanations (SHAP), and functional ANOVA. However, these approaches are vulnerable to issues like signal cancellation and erroneous off-support extrapolation when strong feature interactions are present.

To address these limitations, we introduce Tensor Separation Learning (TSL), a novel regression model. TSL constructs a sum of rank-1 products derived from univariate functions associated with individual features, utilizing a stagewise greedy algorithm complemented by orthogonal refitting. By mandating separability, the TSL framework circumvents the information degradation typically caused by additive projections, which often marginalize higher-order interactions.

A key advantage of the TSL model is its complete reconstructibility from first-order partial dependence functions, modulo constant factors. This structural alignment guarantees that the resulting visualizations accurately reflect the fitted components. Furthermore, we provide approximation-rate guarantees for functions possessing bounded mixed $p$-th order partial derivatives. Our empirical results demonstrate that TSL performs competitively against black-box models across various regression benchmarks.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC