Title: Converting Prompts into Model Weights

Abstract: Recent studies indicate that large language models can be effectively steered during inference by altering their internal parameters, specifically via adjustments to activation vectors or weight matrices. Although these methods are potent, they frequently rely on empirical rules, such as generating "steering vectors" based on the mean activations of contrasting prompts. Expanding on the foundational research of Dherin et al. (2025)—who established that a prompt’s impact corresponds to implicit, token-dependent weight updates and proposed the initial notion of a static thought patch for compressing prompts—we refine this concept into a rigorous algorithm for direct model modification. We present a systematic approach to distill this fleeting data into token-independent thought vectors and matrices. These components not only offer a theoretical underpinning for current vector- and matrix-based editing strategies but also provide a direct, computation-based pathway for converting text inputs into reusable weight modifications, facilitating complex architectural adjustments and the integration of new knowledge.

Source: arXiv Generated at: 2026-06-04 00:00:00 UTC