Filtered by themeMeasuring emotion20 papersclear ✕

Emotions on the Go: Mobile Emotion Assessment in Real-Time using Facial ExpressionsThomas Kosch · 2020Exploiting emotions for user interface evaluation became an increas- ingly important research objective in Human-Computer Interaction. Emotions are usually assessed through surveys that do not allow information to be collected in real-time. In our work, we suggest the use of smartphones for mobile emotion assessment. We use the front-facing smartphone camera as a tool for emotion detection based on facial expressions. Such information can be used to reflect on emotional states or provide emotion-aware user interface adapta- tion. We collected facial expressions along with app usage data in a two-week field study consisting of a one-week training phase and a one-week testing phase. We built and evaluated a person-dependent classifier, yielding an average classification improvement of 33% compared to classifying facial expressions only. Furthermore, we correlate the estimated emotions with concurrent app usage to draw insights into changes in mood. Our work is complemented by a discussion of the feasibility of probing emotions on-the-go and potential use cases for future emotion-aware applications. CCS CONCEPTS • Human-centered computing →Smartphones; Mobile devices; Ubiquitous and mobile computing design and evaluation methods.

UX of AI & PersonalizationResearch Papers

Seemo: A Computational Approach to See EmotionsZhe Liu, Anbang Xu et al. · 2018Successful human interactions are based on becoming aware of other’s emotion and making adaptations accordingly. However, understanding emotion is a complex task that has generated countless debates among researchers over the past decades. The abstractive nature of human emotion highlights the need for a new data-driven approach that can better describe and compare across fine-grained emotional states. In this study, we propose Seemo, a novel neural embedding framework, which allows us to map human emotions into vector space representations. Seemo is trained using Twitter data and is evaluated on two fundamental use cases in traditional emotion research: determining the underlying dimensions of emotions and identifying the set of basic emotions. The evaluation reveals that on both tasks Seemo can generate results consistent with the mainstream theories. Results also show that the vector space representation of Seemo can effectively decode the important relationships between emotions that were usually not explicitly presented. Author Keywords Emotion; affective computing; social computing; computational social science ACM Classification Keywords H.5.2. Information Interfaces and Presentation: Group and Organization Interfaces

UX of AI & PersonalizationResearch Papers

Wearable ESM: Differences in the Experience Sampling Method across Wearable Devices2016The Experience Sampling Method is widely used for collecting self-report responses from people in natural settings. While most traditional approaches rely on using a phone to trigger prompts and record information, wearable devices now offer new opportunities that may improve this method. This research quantitatively and qualitatively studies the experience sampling process on head-worn and wrist-worn wearable devices, and compares them to the traditional “smartphone in the pocket.” To enable this work, we designed and implemented a custom application to provide similar prompts across the three types of devices and evaluated it with 15 individuals for five days (75 days total), in the context of real-life stress measurement. We found significant differences in response times across devices, and captured tradeoffs in interaction types, screen size, and device familiarity that can affect both users’ experience and the reports made by users. Author Keywords Experience Sampling Method; Ecological Momentary Assessment; wearable devices; smartwatch; smartphone; smart-eyewear; Google Glass. ACM Classification Keywords H.5.2. Information interfaces and presentation: User Interfaces.

UX of AI & PersonalizationResearch Papers

Review of eye tracking metrics involved in emotional and cognitive processesVasileios Skaramagkas, Giorgos Giannakakis et al.Eye behaviour provides valuable information reveal- ing one’s higher cognitive functions and state of affect. Although eye tracking is gaining ground in the research community, it is not yet a popular approach for the detection of emotional and cognitive states. In this paper, we present a review of eye and pupil tracking related metrics (such as gaze, fixations, saccades, blinks, pupil size variation, etc.) utilized towards the detection of emotional and cognitive processes, focusing on visual attention, emotional arousal and cognitive workload. Besides, we investigate their involvement as well as the computational recognition meth- ods employed for the reliable emotional and cognitive assessment. The eye-tracking publicly available datasets employed in relevant research efforts were concentrated and described their specifi- cations and details. The multimodal approaches which combine eye-tracking features with other modalities (e.g. biosignals), along with artificial intelligence and machine learning techniques were also surveyed in terms of their recognition/classification accuracy. The limitations, current open research problems and prospective future research directions were discussed for the usage of eye- tracking as the primary sensor modality. This study aims to comprehensively present the most robust and significant eye/pupil metrics based on available literature towards the development of a robust emotional or cognitive computational model. Index Terms—eye tracking, gaze, pupil, fixations, saccades, smooth pursuit, blinks, stress, visual attention, emotional arousal, cognitive workload, emotional arousal datasets, cognitive work- load datasets Vasileios Skaramagkas, Giorgos Giannakakis, Emmanouil Ktistakis, Dim- itris Manousos, Ioannis Karatzanis are with the Institute of Computer Science, Foundation for Research and Technology Hellas (FORTH), GR-700 13 Heraklion, Crete, Greece (Email: vskaramag@ics.forth.gr, ggian@ics.forth.gr, mandim@ics.forth.gr, karatzan@ics.forth.gr) Giorgos Giannakakis is with the Institute of AgriFood and Life Sciences, University Research Centre, Hellenic Mediterranean University, Heraklion, Greece. Emmanouil Ktistakis is with the Institute of Computer Science, Foundation for Research and Technology Hellas (FORTH) and the Laboratory of Optics and Vision, School of Medicine, University of Crete, Heraklion, Greece (Email: mankti@ics.forth.gr) Nikolaos S. Tachos and Evanthia E. Tripoliti, are with the Department of Biomedical Rese

Emotional DesignResearch Papers

Classifying Emotional States through EEG-Derived Spectrograms2024Identifying emotional state of a person plays an important role in a multitude of applications such as affective computing, human-computer interaction and most importantly healthcare. Understanding and correctly identifying human emotions can improve mental health assessments, making it possible to obtain personalized treatment plans and increase the user experience across a variety of digital applications. Our work explores the feasibility of emotion classification using EEG signals recorded from multiple users while experiencing various emotions. Working with time series data of the brain activity in various participants, who are experiencing particular emotions over an interval of 15 seconds. To train and evaluate the classification model several kinds of machine learning and deep learning models such as CNN, RNN and LSTM, are employed. We compare general sequential models with image processing models in the task of classifying signal data. Observing the model’s ability to generalize across the population.

Emotional DesignResearch Papers

The Future of Emotion in Human-Computer InteractionGreg Wadley · 2022Emotion has been studied in HCI for two decades, with specifc tra­ ditions interested in sensing, expressing, transmitting, modelling, experiencing, visualizing, understanding, constructing, regulating, manipulating or adapting to emotion in human-human and human- computer interactions. This CHI 2022 workshop on the Future of Emotion in Human-Computer Interaction brings together in­ terested researchers to take stock of research on emotion in HCI to-date and to explore possible futures. Through group discussion and collaborative speculation we will address questions such as: What are the relationships between digital technology and human emotion? What roles does emotion play in HCI research? How should HCI researchers conceptualize emotion? When should HCI researchers use interdisciplinary theories of emotion or create new theory? Can specifc emotions be designed for, and where is this knowledge likely to be applied? What are the implications of emo­ tion research for design, ethics and wellbeing? What is the future of emotion in human-computer interaction? CCS CONCEPTS • Human-centered computing → Human computer interaction (HCI).

Emotional DesignResearch Papers

Making Sense of Emotion-Sensing: Workshop on Quantifying Human EmotionsBenjamin Tag · 2021The global pandemic and the uncertainty if and when life will return to normality have motivated a series of studies on human mental health. This research has elicited evidence for increasing numbers of anxiety, depression, and overall impaired mental well-being. But, the global COVID-19 pandemic has also created new opportunities for research into quantifying human emotions: remotely, contact- less, in everyday life. The ubiquitous computing community has long been at the forefront of developing, testing, and building user- facing systems that aim at quantifying human emotion. However, rather than aiming at more accurate sensing algorithms, it is time to critically evaluate whether it is actually possible and in what ways it could be beneficial for technologies to be able to detect user emotions. In this workshop, we bring together experts from the fields of Ubiquitous Computing, Human-Computer Interaction, and Psychology to - long-overdue - merge their expertise and ask the fundamental questions: how do we make sense of emotion-sensing, can and should we quantify human emotions? CCS CONCEPTS • Human-centered computing →Human computer interac- tion (HCI); Ubiquitous and mobile computing; • Applied com- puting →Psychology. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. UbiComp-ISWC ’21 Adjunct, September 21–26, 2021, Virtual, USA © 2021 Copyright held by the owner/author(s). Publication rights licensed to ACM. ACM ISBN 978-1-4503-8461-2/21/09...$15.00 https://doi.org/10.1145/3460418.3479272

Emotional DesignResearch Papers

Research on the Influence of Visual Factors on Emotion Regulation Interaction2021To guide the design direction of emotion regulation products that improve the positive emotions of users, investigation into the correlation between relevant visual factors and multi-dimensional complex emotions is needed. In the present study, an extended product emotion measurement method was adopted to describe the multi-dimensional emotional set of each influencing factor and calculate their weight according to the order. The positive and negative emotion indicators of all influencing factors were compared and the evaluation and ranking factors that affect users' emotional value of emotion regulation products were analyzed. The experimental results reveal that specific emotion mapping scenes on positive emotion are the most significant among the key factors affecting user emotion. Further, the influence of emotional stickers, interactive data visualization, and text on positive emotions decreased in turn. The influence of emotional text on positive emotion was the lowest. Through investigating the visual factors that affect the psychological emotions of users, the development of emotion regulating products could be guided in a more scientific and reasonable manner.

Emotional DesignResearch Papers

Can emotional design really evoke emotion in multimedia learning?2020The aim of this study was to investigate the effects of emotional design on learners’ performance and emotions. If emotional design has any effects on learners’ performance and emotions, then we attempted to explore dimensions that are accounted for such effects. 316 Chinese university students participated in this study. The students were divided into two groups: NED group - students in this group learned material based on neutral design, and PED group - students in this group learned material based on positive design. We explored the differences between two groups in student learning performance as well as emotion. Prior knowledge, retention, and transfer tests were also carried out and outcomes were compared across two groups. In addition, student emotion was measured using a positive affect scale and biofeedback instruments. The results showed that PED group outperformed NED group on retention and transfer tests. However, insignificant difference was found between two groups on student emotion. Our results suggest that, although the effect of emotional design was insignificant on student emotion, it was significant on student learning performance (i.e. on retention and transfer tests). Based on our results, several implications were drawn and suggestions were made for educators and researchers.

Emotional DesignResearch Papers

Emotion detection: a technology review2017Emotion detection has become one of the most important aspects to consider in any project related to Affective Computing. Due to the almost endless applications of this new discipline, the development of emotion detection technologies has brought up as a quite profitable opportunity in the corporate sector. Many start-up enterprises have emerged in the last years, dedicated almost exclusively to a specific type of emotion detection technology. In this paper, we present a thorough review of current technologies to detect human emotions. To this end, we explore the different sources from which emotions can be read, along with existing technologies developed to recognize them. We also explore some application domains in which this technology has been applied. This survey has let us identify the strengths and shortcomings of current technology for emotion detection. We conclude the survey highlighting the aspects that requires further research and development. CCS CONCEPTS • Human-Centered Computing → Interaction Design.

Emotional DesignResearch Papers

The Discrete Emotions Questionnaire: A New Tool for Measuring State Self-Reported EmotionsCindy Harmon-Jones1, Brock Bastian et al. · 2016Several discrete emotions have broad theoretical and empirical importance, as shown by converging evidence from diverse areas of psychology, including facial displays, develop- mental behaviors, and neuroscience. However, the measurement of these states has not progressed along with theory, such that when researchers measure subjectively experi- enced emotions, they commonly rely on scales assessing broad dimensions of affect (posi- tivity and negativity), rather than discrete emotions. The current manuscript presents four studies that validate a new instrument, the Discrete Emotions Questionnaire (DEQ), that is sensitive to eight distinct state emotions: anger, disgust, fear, anxiety, sadness, happiness, relaxation, and desire. Emotion theory supporting the importance of distinguishing these specific emotions is reviewed.

Emotional DesignResearch Papers

The Potential of Virtual Reality for the Investigation of AweAnna Esposito · 2016The emotion of awe is characterized by the perception of vastness and a need for accommodation, which can include a positive and/or negative valence. While a number of studies have successfully manipulated this emotion, the issue of how to elicit particularly intense awe experiences in laboratory settings remains. We suggest that virtual reality (VR) is a particularly effective mood induction tool for eliciting awe. VR provides three key assets for improving awe. First, VR provides users with immersive and ecological yet controlled environments that can elicit a sense of "presence," the subjective experience of "being there" in a simulated reality. Further, VR can be used to generate complex, vast stimuli, which can target specific theoretical facets of awe. Finally, VR allows for convenient tracking of participants' behavior and physiological responses, allowing for more integrated assessment of emotional experience. We discussed the potential and challenges of the proposed approach with an emphasis on VR's capacity to raise the signal of reactions to emotions such as awe in laboratory settings.

Emotional DesignResearch Papers

Comparison of selected off-the-shelf solutions forGrzegorz Brodny, Agata Kołakowsk et al. · 2016The paper concerns accuracy of emotion recognition from facial expressions. As there are a couple of ready off-the-shelf solutions available in the market today, this study aims at practical evaluation of selected solutions in order to provide some insight into what potential buyers might expect. Two solutions were compared: FaceReader by Noldus and Xpress Engine by QuantumLab. The performed evaluation revealed that the recognition accuracies differ for photo and video input data and therefore solutions should be matched to the specificity of the application domain.

Emotional DesignResearch Papers

EMOTION MONITORING – VERIFICATION OF PHYSIOLOGICAL CHARACTERISTICS MEASUREMENT PROCEDURES Agnieszka Landowska2014This paper concerns measurement procedures on an emotion monitoring stand designed for tracking human emotions in the Human-Computer Interaction with physiological characteristics. The paper addresses the key problem of physiological measurements being disturbed by a motion typical for human-computer interaction such as keyboard typing or mouse movements. An original experiment is described, that aimed at practical evaluation of measurement procedures performed at the emotion monitoring stand constructed at GUT. Different locations of sensors were considered and evaluated for suitability and measurement precision in the Human- Computer Interaction monitoring. Alternative locations (ear lobes and forearms) for skin conductance, blood volume pulse and temperature sensors were proposed and verified. Alternative locations proved correlation with traditional locations as well as lower sensitiveness to movements like typing or mouse moving, therefore they can make a better solution for monitoring the Human-Computer Interaction.

Emotional DesignResearch Papers

Emotion recognition and its applications2014The paper proposes a set of research scenarios to be applied in four domains: software engineering, website customization, education and gaming. The goal of applying the scenarios is to assess the possibility of using emotion recognition methods in these areas. It also points out the problems of defining sets of emotions to be recognized in different applications, representing the defined emotional states, gathering the data and training. Some of the scenarios consider possible reactions of an affect-aware system and its impact on users.

Emotional DesignResearch Papers

Emotional Design; Application of a Research-Based Design Approach2006In this paper, we discuss an approach to ‘design for wow’ that focuses on the emotions that constitute a wow-experience. In this approach, the eliciting conditions of these emotions are used to define a product character with a high wow-impact. In addition to the approach, a measurable wow-index is introduced. First, a concept of wow is described in which wow is explained as a combination of fascination, pleasant surprise, and desire. The eliciting conditions of these three emotions are examined and combined to a ‘wow-appraisal.’ This concept is applied in the design of a mobile telephone. A combination of qualitative and quantitative research methods was used to formulate concern themes relevant for the wow-appraisal. These themes were used to create a layered product character and a prototype of the final design. An evaluation study demonstrated that the product designed with this approach rated higher on the wow-index than alter- native products.

Emotional DesignResearch Papers

Affective computing: challenges2003A number of researchers around the world have built machines that recognize, express, model, communicate, and respond to emotional information, instances of ‘‘affective computing.’’ This article raises and responds to several criticisms of affective computing, articulating state-of-the art research challenges, especially with respect to affect in human- computer interaction. r 2003 Elsevier Science Ltd. All rights reserved.

Emotional DesignResearch Papers

Emotion in Human–Computer InteractionScott Brave · 2002Emotion is a fundamental component of being human. Joy, hate, anger, and pride, among the plethora of other emotions, motivate action and add meaning and richness to virtually all human experience. Traditionally, human–computer interaction (HCI) has been viewed as the "ultimate" exception; users must discard their emotional selves to work efficiently and rationality with computers, the quintessentially unemotional artifact. Emotion seemed at best marginally relevant to HCI and at worst oxymoronic. Recent research in psychology and technology suggests a different view of the relationship between humans, computers, and emotion. After a long period of dormancy and confusion, there has been an explosion of research on the psychology of emotion (Gross, 1999). Emotion is no longer seen as limited to the occasional outburst of fury when a computer crashes inexplicably, excitement when a video game character leaps past an obstacle, or frustration at an incomprehensible error message. It is now understood that a wide range of emotions plays a critical role in every computer-related, goal-directed activity, from developing a three-dimensional (3D) CAD model and running calculations on a spreadsheet, to searching the Web and sending an e-mail, to making an online purchase and playing solitaire. Indeed, many psychologists now argue that it is impossible for a person to have a thought or perform an action without engaging, at least unconsciously, his or her emotional systems (Picard, 1997b).

Emotional DesignResearch Papers

The role of emotions in marketingElyria Kemp, Steven W. Kopp · 1999*Emotions are mental states of readiness that arise from appraisals of events or one ' s own thoughts. In this article, the authors discuss the differentiation of emotions from affect, moods, and attitudes, and outline an appraisal theory of emotions. Next, various measurement issues are considered. This is followed by an analysis of the role of arousal in emotions. Emotions as markers, mediators, and moderators of consumer responses are then analyzed. The authors turn next to the influence of emotions on cognitive processes, which is followed by a study of the implications of emotions for volitions, goal-directed behavior, and decisions to help. Emotions and customer satisfaction are briefly explored, too. The article closes with a number of questions for future research.*

Emotional DesignResearch Papers

Mapping the Passions: Toward a High-Dimensional Taxonomy of Emotional Experience and Expression1985What would a comprehensive atlas of human emotions include? For 50 years, scientists have sought to map emotion- related experience, expression, physiology, and recognition in terms of the “basic six”—anger, disgust, fear, happiness, sadness, and surprise. Claims about the relationships between these six emotions and prototypical facial configurations have provided the basis for a long-standing debate over the diagnostic value of expression (for review and latest installment in this debate, see Barrett et al., p. 1). Building on recent empirical findings and methodologies, we offer an alternative conceptual and methodological approach that reveals a richer taxonomy of emotion. Dozens of distinct varieties of emotion are reliably distinguished by language, evoked in distinct circumstances, and perceived in distinct expressions of the face, body, and voice. Traditional models—both the basic six and affective-circumplex model (valence and arousal)—capture a fraction of the systematic variability in emotional response. In contrast, emotion- related responses (e.g., the smile of embarrassment, triumphant postures, sympathetic vocalizations, blends of distinct expressions) can be explained by richer models of emotion. Given these developments, we discuss why tests of a basic-six model of emotion are not tests of the diagnostic value of facial expression more generally. Determining the full extent of what facial expressions can tell us, marginally and in conjunction with other behavioral and contextual cues, will require mapping the high-dimensional, continuous space of facial, bodily, and vocal signals onto richly multifaceted experiences using large-scale statistical modeling and machine-learning methods.

Emotional DesignResearch Papers