Title: Language Model Networks: Supervision-Efficient Learning through Dense Communication

Abstract:

Language models are increasingly integrated not merely as independent predictors but as integral components within broader inference architectures, ranging from test-time scaling mechanisms to multi-agent collaborative systems. This paper investigates language model networks, a framework where pre-trained models function as reusable nodes, with emergent intelligence arising from their structural topology, communication patterns, and optimization strategies. While current systems predominantly rely on natural language for communication—offering ease of deployment—they suffer from being discrete, inefficient, and difficult to optimize via end-task supervision. To address these limitations, we introduce LMNet, a dense and differentiable instantiation of this paradigm. LMNet employs stripped large language models (LLMs) as vertex modules and trainable sequence-to-sequence modules as communication edges. This architecture allows intermediate nodes to exchange dense vectors while maintaining natural-language interfaces at the system boundaries. By eliminating intermediate embedding and de-embedding steps, LMNet facilitates efficient information transfer, supports end-to-end gradient optimization, and enables learned communication protocols that surpass hand-designed alternatives. Experimental results demonstrate that this approach yields significant performance gains with minimal additional training overhead and exhibits robust adaptation capabilities even under limited supervision.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC