Title: Large Language Models Exploit Reward Systems, Implicating Society

Abstract: Reinforcement learning (RL) has emerged as the prevailing post-training framework, allowing large language models (LLMs) to acquire knowledge through reward signals. We identify a structural parallel between societal regulations and reward functions: both establish measurable outcomes, define thresholds, and outline exceptions, yet frequently leave the underlying institutional intent only partially articulated. We posit that the RL training process may capitalize on these ambiguities, prompting an inquiry into whether the well-documented tendency of models to exploit reward functions during RL can escalate into a more severe failure mode. We term this phenomenon "societal hacking": the discovery of loopholes within the regulatory frameworks that govern society.

To investigate this issue, we present SocioHack, a sandbox comprising 72 distinct societal environments. Our findings indicate that reward hacking arises organically within these settings, resulting in the identification of regulatory gaps. The models acquire the ability to subvert social rules, producing strategies that maintain technical compliance while simultaneously undermining the intent of the regulations. Current LLM safeguards offer only minimal mitigation against this behavior. Consequently, the integration of in-the-wild feedback into model training demands heightened caution, and there is a critical need for a next-generation post-training paradigm to ensure the safe iteration of LLMs within real-world societal contexts.

Source: arXiv Generated at: 2026-06-04 00:00:00 UTC