Emotional Cause Analysis Based on In-Context Learning of Large Language Models

Abstract

The ability to understand emotions is an essential component of human-like artificial intelligence, as emotions greatly influence human cognition, decision making, and social interactions. In addition to emotion recognition in conversations, the task of identifying the potential causes behind an individual’s emotional state in conversations is of great importance in many application scenarios. The main content of our research is Multimodal Emotion Cause Analysis in Conversations (MECAC), which aims at extracting all pairs of emotions and their corresponding causes from conversations. Under different modality settings, it consists of two specific circumstances: Textual Emotion-Cause Pair Extraction in Conversations (TECPE) and Multimodal Emotion-Cause Pair Extraction in Conversations (MECPE). For this shared study, the main dataset used is a multimodal emotion cause dataset ECF 2.0 sourced from the sitcom Friends. This dataset contains 1,715 conversations and 16,720 utterances, where 12,256 emotion-cause pairs are annotated at the utterance level, covering three modalities (language, audio, and vision). We conducted follow-up research on this benchmark dataset, using mainstream large language models (LLMs) such as GPT3.5, GPT4V, Llama2, Llama3 and Flan-T5. We achieved remarkable results for this challenging task in the multimodel emotion cause analysis field. For Textual Emotion-Cause Pair Extraction in Conversations (TECPE), we achieved an emotional cause extraction result of ACC cause 0.5708 and F1 cause 0.3243. For Multimodal Emotion-Cause Pair Extraction in Conversations (MECPE), we achieved an emotional cause extraction result of w-avg. Strict F1 0.3982 and Strict F1 0.4017.