Title: BehaviorBench: Simulating Authentic User Choices via Behavioral Data

Abstract:

While decision-support systems increasingly aim to tailor their outputs to individual users, there is a notable scarcity of robust evaluation data for this challenge. Current benchmarks for assessing user comprehension frequently depend on simulated agents or behavior generated by models, despite emerging research warning that such model-driven simulations may systematically drift away from actual human conduct. To address this gap, we present \textsc{BehaviorBench}, a novel benchmark designed to assess personalized decision modeling using authentic behavioral traces.

\textsc{BehaviorBench} constructs detailed decision histories at the wallet level by analyzing public on-chain records and prediction-market data. These histories are structured into two distinct, complementary task layers: \emph{Belief prediction}, which forecasts a user’s ultimate stance and level of confidence in a given market, and \emph{Trade prediction}, which anticipates the specific direction and volume of individual transactions. The dataset encompasses 2,000 evaluation wallets, yielding 141,445 instances for Belief prediction and 1,485,972 instances for Trade prediction, utilizing disjoint support pools to facilitate retrieval-based evaluation.

We tested both frontier and open-weight generative models across four different history interfaces: no personalization, direct access to recent history, generated user profiles, and retrieved evidence from support wallets. Our findings indicate that personalization yields more consistent improvements for Belief prediction than for Trade prediction. Furthermore, model performance rankings shift depending on the task layer and metrics used, with varying history interfaces revealing distinct failure modes. \textsc{BehaviorBench} establishes a critical evaluation framework for determining whether personalized approaches can effectively leverage real-world behavioral evidence, moving beyond reliance on simulated users alone.

Source: arXiv Generated at: 2026-06-03 00:00:00 UTC