Title: Efficient and Accurate Function Vectors

Abstract: Function vectors (FVs), which serve as task representations extracted during in-context learning, offer a mechanism to guide Large Language Models (LLMs). Despite their utility, the specific design parameters involved in their construction have received limited attention. This study investigates how different definitions of FVs for instructions perform across two key variables: the selection of attention heads and the method of steering. Regarding head selection, employing gradient-based attributions via Layer-wise Relevance Propagation (LRP) leads to significant gains in both efficiency and accuracy. In terms of steering, implementing a distributed approach results in superior accuracy when compared to straightforward aggregation techniques. The source code for this work is publicly accessible.

Source: arXiv Generated at: 2026-06-04 00:00:00 UTC