Title: Anchoring Functional Similarity Through Invariance-Aware Model Stitching

Abstract:

In the field of deep learning, the assessment of functional similarity measures the degree to which models, trained independently, acquire comparable input-output mappings. Within the context of model stitching, this concept is defined as representation forward compatibility—specifically, the capacity of two models’ representations to be aligned for task resolution. However, recent research identifies a significant flaw in current methodologies: models that depend on distinct informational cues can still yield compatible representations, leading to artificially inflated perceptions of similarity (Smith et al., 2025). We argue that this discrepancy arises because conventional model stitching fails to account for the invariance properties inherent in the stitched architectures. To overcome this issue, we propose an invariance-aware model stitching framework governed by a forward–backward compatibility requirement. By examining various stitching configurations, we investigate the dynamic between forward and backward compatibility, demonstrating that our approach offers a more rigorous method for evaluating functional similarity and uncovers functional differences that were previously hidden.

Source: arXiv Generated at: 2026-06-03 00:00:00 UTC