Filtered by themeEmotion recognition23 papersclear ✕

A Novel Emotion-Aware Networking Model for Enhanced User Experience in 5G networksViktor Stoynov · 2025The emergence of 5G networks has considerably broadened the capabilities and features of mobile networks. A notable development in this area is the incorporation of emotional intelligence into mobile networks, leading to the proposal of the Emotion-aware Networking Paradigm (ENP) in this paper. ENP strives to equip mobile networks with the ability to identify and understand user emotions and adapt services accordingly. This paper introduces a model for implementing ENP in 5G mobile networks, called the Emotion-aware Networking Model (ENM), detailing its structure, functions, and main characteristics. This paper also explores the challenges and prospects related to ENM and offers examples of potential use cases. Additionally, this paper presents the Emotion-Aware Service and User Profile Management Function (EASUPMF) for ENM, designed to unleash ENM's potential by managing user profiles and delivering personalized emotional services based on emotion recognition. Several new Key Performance Indicators (KPIs) are also established to assess ENM's performance, such as Emotional Diversity, Emotional Alignment, and Emotional Intensity. Our research indicates that ENM could transform the mobile network sector and enhance user experiences by delivering tailored services grounded in user emotions.

UX of AI & PersonalizationResearch Papers

Using user emotions to trigger UI adaptationUniv. Grenoble Alpes · 2025User experience is an important user interface quality factor that needs to be maintained during interaction. With this goal, we propose to adapt user interfaces to users’ emotions. This requires being able to detect at runtime negative emotions to trigger adaptations. However, emotions depend on users’ gender and age. This makes it difficult to identify negative emotions and their cause. This paper studies the categorization of emotions depending on users’ age and gender and two user experience dimensions (aesthetics and usability). Thanks to an experiment, it proposes some emotions thresholds that can be used for detecting usability and aesthetics problems from users’ age and gender. These results are used in a user interface adaptation system to infer negative emotions and user interface problems.

UX of AI & PersonalizationResearch Papers

Utilizing emotion recognition technology to enhance user experience in real- timeYuanyuan Xu · 2024In recent years, advancements in human-computer interaction (HCI) have led to the emergence of emotion recognition technology as a crucial tool for enhancing user engagement and satisfaction. This study investigates the application of emotion recognition technology in real-time environments to monitor and respond to users’ emotional states, creating more personalized and intuitive interactions. The research employs convolutional neural networks (CNN) and long short-term memory networks (LSTM) to analyze facial expressions and voice emotions. The experimental design includes an experimental group that uses an emotion recognition system, which dynamically adjusts learning content based on detected emotional states, and a control group that uses a traditional online learning platform. The results show that real-time emotion monitoring and dynamic content adjustments significantly improve user experiences, with the experimental group demonstrating better engagement, learning outcomes, and overall satisfaction. Quantitative results indicate that the emotion recognition system reduced task completion time by 14.3%, lowered error rates by 50%, and increased user satisfaction by 18.4%. These findings highlight the potential of emotion recognition technology to enhance user experiences. However, challenges such as the complexity of multimodal data integration, real-time processing capabilities, and privacy and data security issues remain. Addressing these challenges is crucial for the successful implementation and widespread adoption of this technology. The paper concludes that emotion recognition technology, by providing personalized and adaptive interactions, holds significant promise for improving user experience and offers valuable insights for future research and practical applications.

UX of AI & PersonalizationResearch Papers

Optimizing Emotion Classification from EEG Signals: A Comparative Analysis of Optimization Techniques on the DEAP DatasetJanki Broker · 2024Electroencephalogram (EEG) signals for emotion classification have gained significant attention in recent times due to its implicit applications in varied fields, including human- computer interaction, affective computing, and substantiated counseling sessions. This study set out to explore the effectiveness of convolutional neural networks (CNNs) in classifying emotions using the DEAP dataset, a publicly available repository of EEG recordings and corresponding emotional annotations by experimenting with different optimizers and loss functions, like "Adamax" and "binary_crossentropy,", etc. By carefully manipulating these optimizers, the study aims to examine and determine the optimal combination with the highest classification accuracy. The findings contribute valuable insights into fine- tuning CNN models for emotion classification based on hyperparameters, paving the way for more robust and effective applications such as personalized diagnosis for counseling, stress- busting techniques and so on. The promising result of this research shows that the “Adamax” optimizer outperforms other optimizers by 82.98 % of the accuracy of classification.

UX of AI & PersonalizationResearch Papers

Toward an Interactive Reading Experience: Deep Learning Insights and Visual Narratives of Engagement and EmotionJayasankar Santhosh, Akshay Palimar Pai et al. · 2024Engagement and emotion are critical components that significantly influence a reader’s experience during a reading task. Despite the crucial role of engagement and emotions in shaping our reading experience, accurately tracking these dynamic states during actual reading remains a significant challenge. This study bridges this gap by detecting engagement and emotion levels during a reading task by leveraging the power of state-of-the-art deep learning models and investigating the correlations between the engagement levels and emotions. An experiment was conducted involving 18 university students reading 14 documents followed by a questionnaire to rate their levels of engagement, valence, and arousal after reading each document. A Tobii 4C eye-tracker with a pro license along with an Empatica E4 wristband were utilized to record behavioral and physiological data from the participants. A range of deep learning models were utilized for computing the engagement, valence, and arousal values, employing both user-independent and user- dependent methods. Our investigation revealed distinct yet complementary strengths in two deep learning models: Transformer excelled in user-independent detection of engagement and emotion with an accuracy of 80.38% (engagement), 71.28% (arousal) and 73.98% (valence) while ResNet shined in the user-dependent setting with an accuracy of 93.56% (engagement), 90.62% (arousal) and 88.70% (valence) which highlights the interplay between individual differences and reading dynamics. Intriguingly, we observed strong, document-specific correlations between engagement and emotion states, suggesting that different texts evoke unique affective responses. We developed an interactive dashboard visualizing predicted engagement and emotions, offering real-time feedback and personalized learning possibilities. The dashboard features an engagement gauge that displays the reader’s level of engagement based on predicted class probabilities, and an emotion emoji serving as a visual cue that illustrates the predicted emotional state of the reader. This technology can inform the design of dynamic interfaces that adjust to individual reading styles and emotional responses, potentially enhancing comprehension and involvement. INDEX TERMS Digital reading, physiological sensing, eye tracking, deep learning, affective state.

UX of AI & PersonalizationResearch Papers

Zero-shot multitask intent and emotion prediction from multimodal data: A benchmark studyMauajama Firdaus b, Dushyant Singh Chauhan c · 2023Empathy involves comprehending and sharing the emotions of another person. In the realm of conversational AI, empathy pertains to the AI’s capacity to understand and respond suitably to the user’s emotions and needs. Conversational AI with empathetic capabilities can heighten the user experience by making interactions more personalized and natural. At present, machine learning algorithms are commonly utilized in existing conversational AI systems to recognize emotions and corresponding empathetic intents from annotated data. Nonetheless, this approach is not without limitations, being expensive and time-consuming. Our present work takes a holistic approach to empathy in conversational AI, where we propose a novel zero-shot multitask framework, the Zero-shot Intent Emotion Detection (ZIED) network, identifies both emotions and intents in a multimodal setting. We developed an end-to-end model that concurrently captures textual, audio, and visual representations and integrates the different modalities using cross-attention mechanisms. Our experimental results, based on the EmoInt-MD dataset, show that incorporating all three modalities results in the best performance for both emotion and empathetic intent detection. We observed a noteworthy improvement of over 6% and 4% for intent and emotion, respectively, for various ratios of seen and unseen classes.

UX of AI & PersonalizationResearch Papers

Personalized recommendation model of electronic commerce in new media era based on semantic emotion analysis2022© 2022 Liu and Ding. This is an open-access article distributed under the terms of the Creative Commons Attribution License \(CC BY\). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

UX of AI & PersonalizationResearch Papers

Deep Learning for Emotion Driven User ExperiencesCarmen Bisogni, Lucia Cascone et al. · 2021Deep knowledge about user characteristics and behaviors opens new and promising landscapes to the User Experience. Thanks to emerging machine learning techniques, a new form of communication chan- nel among users and applications may be exploited for customizing and finely tuning the dynamic behav- ior of applications to the peculiarities of their users. This work investigates the empathic improvement of the User Experiences and exploits inferences on user expressions for activating gaming and entertain- ment events, that are adopted in a cinematic way to create dynamic application behaviors. The presented approach is applied to a third/first-person horror adventure and a classic table game. As already verified in a preliminary phase of the research, user impressions, collected as subjective evaluation in a controlled experiment, are positive and encourage further steps, like the evaluation of other inferences on users. © 2021 Published by Elsevier B.V.

UX of AI & PersonalizationResearch Papers

Emotions on the Go: Mobile Emotion Assessment in Real-Time using Facial ExpressionsThomas Kosch · 2020Exploiting emotions for user interface evaluation became an increas- ingly important research objective in Human-Computer Interaction. Emotions are usually assessed through surveys that do not allow information to be collected in real-time. In our work, we suggest the use of smartphones for mobile emotion assessment. We use the front-facing smartphone camera as a tool for emotion detection based on facial expressions. Such information can be used to reflect on emotional states or provide emotion-aware user interface adapta- tion. We collected facial expressions along with app usage data in a two-week field study consisting of a one-week training phase and a one-week testing phase. We built and evaluated a person-dependent classifier, yielding an average classification improvement of 33% compared to classifying facial expressions only. Furthermore, we correlate the estimated emotions with concurrent app usage to draw insights into changes in mood. Our work is complemented by a discussion of the feasibility of probing emotions on-the-go and potential use cases for future emotion-aware applications. CCS CONCEPTS • Human-centered computing →Smartphones; Mobile devices; Ubiquitous and mobile computing design and evaluation methods.

UX of AI & PersonalizationResearch Papers

Seemo: A Computational Approach to See EmotionsZhe Liu, Anbang Xu et al. · 2018Successful human interactions are based on becoming aware of other’s emotion and making adaptations accordingly. However, understanding emotion is a complex task that has generated countless debates among researchers over the past decades. The abstractive nature of human emotion highlights the need for a new data-driven approach that can better describe and compare across fine-grained emotional states. In this study, we propose Seemo, a novel neural embedding framework, which allows us to map human emotions into vector space representations. Seemo is trained using Twitter data and is evaluated on two fundamental use cases in traditional emotion research: determining the underlying dimensions of emotions and identifying the set of basic emotions. The evaluation reveals that on both tasks Seemo can generate results consistent with the mainstream theories. Results also show that the vector space representation of Seemo can effectively decode the important relationships between emotions that were usually not explicitly presented. Author Keywords Emotion; affective computing; social computing; computational social science ACM Classification Keywords H.5.2. Information Interfaces and Presentation: Group and Organization Interfaces

UX of AI & PersonalizationResearch Papers

Designing Responsive Interactive Applications by Emotion-Tracking and Pattern-Based Dynamic User Interface AdaptationSanim Rashi, Christian Herdin · 2016Model-based user interface development environments (MB-UIDEs) can be enhanced by pattern-based frameworks to allow for richer design capa‐ bilities and more flexible responsive behavior during the runtime of the imple‐ mented target application. In this paper an experimental system prototype for integrating facial analysis and eye-tracking into a pattern-based dynamic user interface adaptation process is discussed. The resulting system evaluates the emotional state of the system users to trigger the HCI-pattern-based adaptation of the user interface. By monitoring the emotional states of the users over longer time periods, while the system changes its behavior and its appearance, conclu‐ sions about the perceived quality dimensions of the interactive application can be drawn.

UX of AI & PersonalizationResearch Papers

Smartphone Accelerometer Data used for Detecting Human EmotionsAndreas Færøvig Olsen, Jim Torresen · 2016The paper outlines work on the classification of emotions using smartphone accelerometer data. Such classi- fication can be used, in conjunction with other methods of emotion detection, to adapt services to the user’s emotional state. The data is collected from individuals who have been carrying their phone in a pocket while walking. An Android app was developed in order to monitor the smartphone accelerometer of the individuals who participated in the study and occasionally requested them to judge and submit their emotional state. This way, data is collected from a natural environment rather than a laboratory setting. The recorded data is then processed and used to train different classifiers to be compared. The machine learning algorithms decision tree, support vector machine and multilayer perceptron are used for this purpose. Emotions are classified in two dimensions: pleasantness and arousal (activation). While the recognition rate for the arousal dimension is promising at 75%, pleasantness is harder to predict, with a recognition rate of 51%. These findings indicate that by only analyzing accelerometer data recorded from a smartphone, it is possible to make predictions of a person’s state of activation.

UX of AI & PersonalizationResearch Papers

Affective modelling of users in HCI using EEGJyotish Kumara, Jyoti kumar · 2016Emotions have potential to play a role in HCI which is primarily dominated by cognitive measures. Human physiological communication channels are dominated by emotions. Emotion affects several human activities like communication, learning, decision making, cognition, perception etc. Further, as emotions are difficult to interpret and hard to measure, technologists and designers have been struggling to incorporate them in design and technology. On the other hand, advancement of technology has both necessitated and enabled us to understand emotions and put them to use in contexts like human computer interaction. This study reports an attempt to model emotions by means of electroencephalography (EEG). Video stimuli of four representative basic emotions based on Navarasa theory of Ancient Indian treatise called Natya Shastra were shown to participants and EEG data was collected. Power spectrum analysis of EEG signals associated with emotions was done. Further, the EEG analysis findings were compared with the subject’s self-reports about their emotional states during the experiment. EEG results have shown significantly consistent frequency patterns across the brain lobes for a given emotion. This study suggests that human emotions can be modeled for use in HCI either as an affect assessment tool or for affect based intelligent interactions. © 2015 The Authors. Published by Elsevier B.V. Peer-review under responsibility of the Scientific Committee of IHCI 2015.

UX of AI & PersonalizationResearch Papers

A Comparative Study on different approaches of Real Time Human Emotion Recognition based on Facial Expression DetectionDey Anurag · 2015Facial Expression Recognition lies in one of the crucial areas of research for human-computer interaction and human emotion identification. For a system to recognize a facial expression it needs to come across multiple variability of human face like color, texture, posture, expression, orientation and so on. The first step to recognize a facial expression of a person with various facial movements of the muscles beneath the eyes, nose and lips are to be detected and further classifying those features by comparing them with a set of trained data values using a good classifier for recognizing the emotion. In this paper a comparative study of the different approaches initiated for automatic real-time facial expression recognition is undertaken along with their benefits and flaws which will further help in developing and improving the system.

UX of AI & PersonalizationResearch Papers

Classifying Emotional States through EEG-Derived Spectrograms2024Identifying emotional state of a person plays an important role in a multitude of applications such as affective computing, human-computer interaction and most importantly healthcare. Understanding and correctly identifying human emotions can improve mental health assessments, making it possible to obtain personalized treatment plans and increase the user experience across a variety of digital applications. Our work explores the feasibility of emotion classification using EEG signals recorded from multiple users while experiencing various emotions. Working with time series data of the brain activity in various participants, who are experiencing particular emotions over an interval of 15 seconds. To train and evaluate the classification model several kinds of machine learning and deep learning models such as CNN, RNN and LSTM, are employed. We compare general sequential models with image processing models in the task of classifying signal data. Observing the model’s ability to generalize across the population.

Emotional DesignResearch Papers

Making Sense of Emotion-Sensing: Workshop on Quantifying Human EmotionsBenjamin Tag · 2021The global pandemic and the uncertainty if and when life will return to normality have motivated a series of studies on human mental health. This research has elicited evidence for increasing numbers of anxiety, depression, and overall impaired mental well-being. But, the global COVID-19 pandemic has also created new opportunities for research into quantifying human emotions: remotely, contact- less, in everyday life. The ubiquitous computing community has long been at the forefront of developing, testing, and building user- facing systems that aim at quantifying human emotion. However, rather than aiming at more accurate sensing algorithms, it is time to critically evaluate whether it is actually possible and in what ways it could be beneficial for technologies to be able to detect user emotions. In this workshop, we bring together experts from the fields of Ubiquitous Computing, Human-Computer Interaction, and Psychology to - long-overdue - merge their expertise and ask the fundamental questions: how do we make sense of emotion-sensing, can and should we quantify human emotions? CCS CONCEPTS • Human-centered computing →Human computer interac- tion (HCI); Ubiquitous and mobile computing; • Applied com- puting →Psychology. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. UbiComp-ISWC ’21 Adjunct, September 21–26, 2021, Virtual, USA © 2021 Copyright held by the owner/author(s). Publication rights licensed to ACM. ACM ISBN 978-1-4503-8461-2/21/09...$15.00 https://doi.org/10.1145/3460418.3479272

Emotional DesignResearch Papers

A Novel User Emotional Interaction Design Model Using Long and Short-Term Memory Networks and Deep Learning2021Emotional design is an important development trend of interaction design. Emotional design in products plays a key role in enhancing user experience and inducing user emotional resonance. In recent years, based on the user's emotional experience, the design concept of strengthening product emotional design has become a new direction for most designers to improve their design thinking. In the emotional interaction design, the machine needs to capture the user's key information in real time, recognize the user's emotional state, and use a variety of clues to finally determine the appropriate user model. Based on this background, this research uses a deep learning mechanism for more accurate and effective emotion recognition, thereby optimizing the design of the interactive system and improving the user experience. First of all, this research discusses how to use user characteristics such as speech, facial expression, video, heartbeat, etc., to make machines more accurately recognize human emotions. Through the analysis of various characteristics, the speech is selected as the experimental material. Second, a speech-based emotion recognition method is proposed. The mel-Frequency cepstral coefficient (MFCC) of the speech signal is used as the input of the improved long and short-term memory network (ILSTM). To ensure the integrity of the information and the accuracy of the output at the next moment, ILSTM makes peephole connections in the forget gate and input gate of LSTM, and adds the unit state as input data to the threshold layer. The emotional features obtained by ILSTM are input into the attention layer, and the self-attention mechanism is used to calculate the weight of each frame of speech signal. The speech features with higher weights are used to distinguish different emotions and complete the emotion recognition of the speech signal. Experiments on the EMO-DB and CASIA datasets verify the effectiveness of the model for emotion recognition. Finally, the feasibility of emotional interaction system design is discussed.

Emotional DesignResearch Papers

Emotional MachinesJordi Vallverdú · 2019Artificial Intelligence (AI) is the motor that fuels the most profound revolution in the history of humankind. The new era of information society [10], even called "the Fourth Revolution" [8], has produced deep changes into sciences, economies, and societies [9]. This revolution is not only about formal heuristics, or the called "algorithmic society", but also is explicitly related to the creation of strong interactions between humans and machines. Consequently, these machines must be able to deal with the most intrinsic feature of human nature: emotions. If emotions had been historically considered as disturbing elements of human rationality, the neurological revolution and new experimental tools such as magnetic scanning of the brain (with great success for fMRI) made possible a complete Copernican turn into the evaluation of the role of emotions in cognitive frameworks [13,14]. This process led experts in natural and artificial cognition to consider the necessity of including emotions into their models [37]. Affective computing or social robotics [7,44] increased not only the necessity of studies about how to improve the emotional interaction between humans and machines [57,59] but also how to design cognitive architectures which included biomimetic elements related with emo-

Emotional DesignResearch Papers

Emotion detection: a technology review2017Emotion detection has become one of the most important aspects to consider in any project related to Affective Computing. Due to the almost endless applications of this new discipline, the development of emotion detection technologies has brought up as a quite profitable opportunity in the corporate sector. Many start-up enterprises have emerged in the last years, dedicated almost exclusively to a specific type of emotion detection technology. In this paper, we present a thorough review of current technologies to detect human emotions. To this end, we explore the different sources from which emotions can be read, along with existing technologies developed to recognize them. We also explore some application domains in which this technology has been applied. This survey has let us identify the strengths and shortcomings of current technology for emotion detection. We conclude the survey highlighting the aspects that requires further research and development. CCS CONCEPTS • Human-Centered Computing → Interaction Design.

Emotional DesignResearch Papers

Comparison of selected off-the-shelf solutions forGrzegorz Brodny, Agata Kołakowsk et al. · 2016The paper concerns accuracy of emotion recognition from facial expressions. As there are a couple of ready off-the-shelf solutions available in the market today, this study aims at practical evaluation of selected solutions in order to provide some insight into what potential buyers might expect. Two solutions were compared: FaceReader by Noldus and Xpress Engine by QuantumLab. The performed evaluation revealed that the recognition accuracies differ for photo and video input data and therefore solutions should be matched to the specificity of the application domain.

Emotional DesignResearch Papers

EMOTION MONITORING – VERIFICATION OF PHYSIOLOGICAL CHARACTERISTICS MEASUREMENT PROCEDURES Agnieszka Landowska2014This paper concerns measurement procedures on an emotion monitoring stand designed for tracking human emotions in the Human-Computer Interaction with physiological characteristics. The paper addresses the key problem of physiological measurements being disturbed by a motion typical for human-computer interaction such as keyboard typing or mouse movements. An original experiment is described, that aimed at practical evaluation of measurement procedures performed at the emotion monitoring stand constructed at GUT. Different locations of sensors were considered and evaluated for suitability and measurement precision in the Human- Computer Interaction monitoring. Alternative locations (ear lobes and forearms) for skin conductance, blood volume pulse and temperature sensors were proposed and verified. Alternative locations proved correlation with traditional locations as well as lower sensitiveness to movements like typing or mouse moving, therefore they can make a better solution for monitoring the Human-Computer Interaction.

Emotional DesignResearch Papers

Emotion recognition and its applications2014The paper proposes a set of research scenarios to be applied in four domains: software engineering, website customization, education and gaming. The goal of applying the scenarios is to assess the possibility of using emotion recognition methods in these areas. It also points out the problems of defining sets of emotions to be recognized in different applications, representing the defined emotional states, gathering the data and training. Some of the scenarios consider possible reactions of an affect-aware system and its impact on users.

Emotional DesignResearch Papers

Affective computing: challenges2003A number of researchers around the world have built machines that recognize, express, model, communicate, and respond to emotional information, instances of ‘‘affective computing.’’ This article raises and responds to several criticisms of affective computing, articulating state-of-the art research challenges, especially with respect to affect in human- computer interaction. r 2003 Elsevier Science Ltd. All rights reserved.

Emotional DesignResearch Papers