Title: AnyEdit++: Enhancing Long-Form Knowledge Editing in LLMs Through Bayesian Surprise

Abstract:

Maintaining generation coherence while editing complex, long-form knowledge remains a formidable hurdle for Large Language Models. While existing autoregressive techniques, such as AnyEdit, help mitigate length limitations, they depend on Fixed-window Chunking. This traditional approach overlooks logical structures, thereby undermining consistency. To overcome these limitations, we introduce AnyEdit++, a structure-aware framework that integrates Bayes-Chunk. This mechanism utilizes Bayesian Surprise to dynamically pinpoint semantic boundaries.

Our approach is grounded in a theoretical framework that outlines two fundamental principles:

1. Structural Independence: We demonstrate that cross-segment interference is significantly reduced when anchor keys are geometrically orthogonal. This condition is inherently met by our surprisal-based boundaries, whereas fixed windows fail to satisfy it.
2. Causal Locality: We show that updates applied at these semantic peaks provide strictly greater control than those introduced at arbitrary split points.

Comprehensive experiments spanning mathematical reasoning, code generation, and narrative tasks reveal that AnyEdit++ outperforms state-of-the-art baselines in both performance and robustness. These results confirm that structural awareness is essential for effective long-form knowledge editing.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC