Title: Mechanism Design Is Not Enough: Prosocial Agents for Cooperative AI

Abstract:

Achieving safe and beneficial interactions among AI agents has become a pivotal concern in contemporary AI safety research. Although mechanism design—defined as the framework for constructing rules that harmonize individual and group goals—can encourage cooperation, it remains uncertain whether this approach alone is adequate to optimize social welfare for Large Language Model (LLM) agents. This study demonstrates that it is not. By leveraging principles from incomplete contract theory, we formally establish that when contracts fail to account for every possible future contingency, a strictly positive welfare loss inevitably occurs, a deficit that no practical mechanism can fully eradicate. However, we identify that agents exhibiting prosocial behavior—those that balance their own interests with the well-being of others—can bridge this gap, yielding outcomes that are both socially optimal and individually advantageous. Our experiments, conducted in multi-agent resource allocation scenarios and classic social dilemmas involving LLMs, confirm that prosociality drives superior results. The conclusion for AI safety is straightforward: fostering scalable cooperative interactions requires more than just well-designed mechanisms; agents must be engineered with intrinsic prosocial tendencies.

Source: arXiv Generated at: 2026-06-03 00:00:00 UTC