Title: T-POP: Test-Time Personalization with Online Preference Feedback

Abstract: Moving beyond the generation of generic responses, tailoring large language models (LLMs) to individual user preferences is an essential advancement. Yet, current personalization techniques struggle with the "cold-start" issue for new users, as they generally depend on either extensive pre-existing user data or slow, resource-heavy fine-tuning processes. To overcome this hurdle, we present a novel framework for real-time personalization that learns from online pairwise preference feedback gathered during the text generation phase. We introduce T-POP (Test-Time Personalization with Online Preference Feedback), an innovative algorithm that merges test-time alignment with dueling bandits. Rather than updating LLM parameters, T-POP directs the decoding process of a frozen model by continuously learning a reward function that reflects user preferences. By utilizing dueling bandits, the system intelligently solicits user input, effectively balancing the exploration of new preferences with the exploitation of acquired knowledge to produce personalized content. Our extensive experiments reveal that T-POP enables rapid and data-efficient personalization, consistently surpassing existing baselines and demonstrating steady improvement as user interactions increase.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC