Title: Integrating Multi-Modal Learning with Genetic Programming: An Examination of Alignment Dynamics in Latent Space Optimization

Abstract:

Symbolic regression (SR) is designed to extract mathematical formulas from datasets, a challenge historically addressed via Genetic Programming (GP) through combinatorial searches across symbolic architectures. Latent Space Optimization (LSO) techniques have emerged as an alternative, employing neural encoders to translate symbolic expressions into continuous domains, thereby shifting the problem from combinatorial search to continuous optimization. SNIP, a contrastive pre-training framework introduced by Meidani et al. (2024) and drawing inspiration from CLIP, represents a significant step forward in LSO. It adopts a multi-modal strategy by aligning symbolic and numeric encoders within a unified latent space. This alignment facilitates the learning of phenotype-genotype mappings, allowing optimization within the numeric domain to indirectly steer the symbolic search.

However, this methodology presupposes fine-grained cross-modal alignment. This assumption contrasts with findings in the literature regarding analogous models such as CLIP, which typically exhibit coarse-grained alignment characteristics. This study critically evaluates whether SNIP fulfills its potential for effective bi-modal optimization in SR. Our experimental results indicate two primary outcomes: first, cross-modal alignment fails to enhance during the optimization process, despite concurrent improvements in fitness; second, the alignment captured by SNIP is insufficiently granular to support principled search efforts within the symbolic space. These insights suggest that while multi-modal LSO offers considerable promise for SR, the practical realization of alignment-guided optimization remains elusive. Consequently, achieving fine-grained alignment stands out as a vital area for subsequent research.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC