Title: Establishing a Theoretical Basis for Self-Play in Theorem Proving Algorithms

Abstract:

Recent studies have demonstrated that self-play, a training methodology allowing models to enhance their own performance, yields promising empirical outcomes in formal theorem proving with Large Language Models (LLMs) (Dong & Ma, 2025). This approach typically employs two collaborative agents: a prover responsible for validating theorems and a conjecturer tasked with generating new theorems to serve as a structured curriculum for the prover. This work introduces a theoretical framework designed to elucidate the self-improvement mechanisms inherent in such self-play algorithms.

First, we represent the domain of theorems as a graph structure, where individual theorems constitute the nodes and edges connect pairs sharing semantic similarities. We then define a set of foundational assumptions that delineate the performance guarantees of a trained prover and specify the conjecturer’s ability to navigate this graph topology.

Second, our analysis demonstrates that if the theorem graph exhibits high connectivity, a prover-conjecturer pair utilizing a reversible random walk for conjecturing is capable of expanding the collection of proved theorems at an exponential rate.

Third, addressing an empirical challenge observed in self-play systems—namely, the tendency of conjecturers to produce overly complex and non-essential theorems—we introduce a metric for assessing the diversity of theorems generated by the conjecturer during training. We propose an enhanced conjecturing algorithm that locally optimizes this diversity metric by calculating the diffusion similarity between adjacent theorems within the graph. Finally, we outline a technique for computing this diffusion similarity, which involves employing contrastive learning to map nodes into Euclidean space and subsequently determining the inner product of their embeddings.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC