Title: Latent Collaboration in Multi-Agent Systems

Abstract:

Multi-agent systems (MAS) represent a significant evolution for large language models (LLMs), shifting the focus from isolated, single-model reasoning to coordinated, system-level intelligence. While current LLM agents typically rely on text-based mediation to facilitate communication and reasoning, this work advances the field by enabling direct collaboration within the continuous latent space. We present LatentMAS, an end-to-end framework that requires no training and facilitates pure latent collaboration among LLM agents.

In the LatentMAS architecture, agents generate latent thoughts auto-regressively using last-layer hidden embeddings rather than textual output. A shared latent working memory is utilized to preserve and transfer these internal representations and latent thoughts between agents, thereby ensuring lossless information exchange without the need for re-encoding. Our theoretical analysis demonstrates that LatentMAS offers superior expressiveness and lossless information preservation compared to standard text-based MAS, all while maintaining lower overall complexity.

Empirical evaluations across nine comprehensive benchmarks—covering areas such as code generation, commonsense understanding, and math and science reasoning—indicate that LatentMAS surpasses both advanced single-agent models and text-based MAS baselines. The framework achieves up to a 14.6% increase in accuracy, reduces output token consumption by 70.8% to 83.7%, and delivers end-to-end inference speeds that are 4 to 4.3 times faster. The code and data for this project are fully open-sourced at https://github.com/Gen-Verse/LatentMAS.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC