Unveiling the Limits of Large Language Models in Inferring Pragmatic Meaning from Non-Verbal Responses
Title: Exposing the Boundaries of Large Language Models in Deriving Pragmatic Meaning from Non-Verbal Cues
Abstract:
While large language models (LLMs) have demonstrated significant advancements in understanding pragmatic language, existing studies have predominantly concentrated on their capacity to interpret verbal interactions. However, non-verbal behavior constitutes a crucial element of human communication, particularly when employed intentionally to transmit indirect messages. This study introduces the inaugural systematic assessment of LLMsā proficiency in deducing pragmatic meaning from dialogues composed entirely of non-verbal responses. We address three primary inquiries: (1) Are LLMs capable of identifying indirect intentions expressed through non-verbal cues? (2) Under what circumstances and via what mechanisms do LLMs miss non-verbal intent? (3) What strategies can enhance LLMsā interpretation of non-verbal intent?
Our findings indicate that LLMs face considerable difficulties in extracting underlying meanings from non-verbal responses, with accuracy rates declining by as much as 60 percentage points relative to verbal counterparts. In-depth analysis further uncovers specific behavioral patterns in how LLMs interpret non-verbal actions and confirms that in-context learning aids in pragmatic inference.
Source: arXiv Generated at: 2026-06-02 00:00:00 UTC
