Filtered by themeExplainability20 papersclear ✕

Article LIME-Mine: Explainable Machine Learning for User Behavior Analysis in IoT ApplicationsXiaobo Cai · 2024In Internet of Things (IoT) applications, user behavior is influenced by factors such as network structure, user activity, and location. Extracting valuable patterns from user activity traces can lead to the development of smarter, more personalized IoT applications and improved user experience. This paper proposes a LIME-based user behavior preference mining algorithm that leverages Explainable AI (XAI) techniques to interpret user behavior data and extract user preferences. By training a black-box neural network model to predict user behavior using LIME and approximating predictions with a local linear model, we identify key features influencing user behavior. This analysis reveals user behavioral patterns and preferences, such as habits at specific times, locations, and device states. Incorporating user behavioral information into the resource scheduling process, combined with a feedback mechanism, establishes an active discovery network of user demand. Our approach, utilizing edge computing capabilities, continuously fine-tunes and optimizes resource scheduling, actively adapting to user perceptions. Experimental results demonstrate the effectiveness of feedback control in satisfying diverse user resource requests, enhancing user satisfaction, and improving system resource utilization.

UX of AI & PersonalizationResearch Papers

A Roadmap of Explainable Artificial Intelligence: Explain to Whom, When, What and How?Ziming Wang · 2024Explainable artificial intelligence (XAI) has gained significant attention, especially in AI-powered autonomous and adaptive systems (AASs). However, a discernible disconnect exists among research efforts across different communities. The machine learning community often overlooks "explaining to whom," while the human-computer interaction community has examined various stakeholders with diverse explanation needs without addressing which XAI methods meet these requirements. Currently, no clear guidance exists on which XAI methods suit which specific stakeholders and their distinct needs. This hinders the achievement of the goal of XAI: providing human users with understandable interpretations. To bridge this gap, this paper presents a comprehensive XAI roadmap. Based on an extensive literature review, the roadmap summarizes different stakeholders, their explanation needs at different stages of the AI system lifecycle, the questions they may pose, and existing XAI methods. Then, by utilizing stakeholders' inquiries as a conduit, the roadmap connects their needs to prevailing XAI methods, providing a guideline to assist researchers and practitioners to determine more easily which XAI methodologies can meet the specific needs of stakeholders in AASs. Finally, the roadmap discusses the limitations of existing XAI methods and outlines directions for future research.

UX of AI & PersonalizationResearch Papers

Predicting the Need for XAI from High-Granularity Interaction DataVagner Figueredo de Santana, Ana Fucs et al. · 2023Recent advances in Artificial Intelligence (AI) and Machine Learning (ML) brought light on the need for explainability in multiple domains (e.g., healthcare, finance, justice, and recruiting). Explainability or Explainable AI (XAI) can be defined as everything that makes AI more understandable to human beings. However, XAI features may vary according to the AI algorithm used. Beyond XAI features, different AI algorithms vary in terms of speed, performance, and costs associated with training/running models. Knowing when to choose the right algorithm for the task at hand, therefore, is fundamental in multiple AI systems, for instance, AutoML and AutoAI. In this paper, we propose a method to analyze patterns of high-granularity user interface (UI) events (i.e., mouse, keyboard, and additional custom events triggered on the millisecond scale) to predict when users will interact with UI elements that provide explainability for the AI in place. In this context, this paper presents: (1) a user study involving 37 participants (7 in the pilot phase and 30 in the main experiment phase) in which people performed a task of reporting a bug using a text form associated with an AI data quality meter and its XAI UI element and (2) an approach to model micro behavior using 𝑛𝑜𝑑𝑒2𝑣𝑒𝑐 to predict when the interaction with XAI UI element will occur. The proposed approach uses a rich dataset (approximately 129k events) and combines 𝑛𝑜𝑑𝑒2𝑣𝑒𝑐and a Logistic Regression classifier. Results obtained show we have obtained an event-by-event prediction of the interaction with XAI with an average F-score of 0.90 (𝜎= 0.06). From the presented results, one expects to support researchers in the realm of UI personalization to consider high-granularity interaction data when predicting the need for XAI while users are interacting with AI model outputs.

UX of AI & PersonalizationResearch Papers

Interpretability, Then What? Editing Machine Learning Models to Reflect Human Knowledge and ValuesAlex Kale · 2022Machine learning (ML) interpretability techniques can reveal unde- sirable patterns in data that models exploit to make predictions— potentially causing harms once deployed. However, how to take action to address these patterns is not always clear. In a collabo- ration between ML and human-computer interaction researchers, physicians, and data scientists, we develop GAM Changer, the first interactive system to help domain experts and data scientists easily and responsibly edit Generalized Additive Models (GAMs) and fix problematic patterns. With novel interaction techniques, our tool puts interpretability into action—empowering users to analyze, val- idate, and align model behaviors with their knowledge and values. Physicians have started to use our tool to investigate and fix pneu- monia and sepsis risk prediction models, and an evaluation with 7 data scientists working in diverse domains highlights that our tool is easy to use, meets their model editing needs, and fits into their current workflows. Built with modern web technologies, our tool runs locally in users’ web browsers or computational notebooks, lowering the barrier to use. GAM Changer is available at the fol- lowing public demo link: https://interpret.ml/gam-changer. CCS CONCEPTS • Computing methodologies →Machine learning.

UX of AI & PersonalizationResearch Papers

Explaining User Models with Different Levels of Detail for Transparent Recommendation: A User StudyMouadh Guesmi · 2022In this paper, we shed light on explaining user models for trans- parent recommendation while considering user personal character- istics. To this end, we developed a transparent Recommendation and Interest Modeling Application (RIMA) that provides interactive, layered explanations of the user model with three levels of detail (basic, intermediate, advanced) to meet the demands of different types of end-users. We conducted a within-subject study (N=31) to investigate the relationship between personal characteristics and the explanation level of detail, and the effects of these two variables on the perception of the explainable recommender system with regard to different explanation goals. Based on the study results, we provided some suggestions to support the effective design of user model explanations for transparent recommendation. CCS CONCEPTS • Human-centered computing →Interactive systems and tools; • Computing methodologies →Artificial intelligence.

UX of AI & PersonalizationResearch Papers

Visual, textual or hybrid: the effect of user expertise on different explanationsMaxwell Szymanski · 2021As the use of AI algorithms keeps rising continuously, so does the need for their transparency and accountability. However, literature often adopts a one-size-fits-all approach for developing explana- tions when in practice, the type of explanations needed depends on the type of end-user. This research will look at user expertise as a variable to see how different levels of expertise influence the under- standing of explanations. The first iteration consists of developing two common types of explanations (visual and textual explana- tions) that explain predictions made by a general class of predictive model learners. These explanations are then evaluated by users of different expertise backgrounds to compare the understanding and ease-of-use of each type of explanation with respect to the different expertise groups. Results show strong differences between experts and lay users when using visual and textual explanations, as well as lay users having a preference for visual explanations which they perform significantly worse with. To solve this problem, the second iteration of this research focuses on the shortcomings of the first two explanations and tries to minimize the difference in understanding between both expertise groups. This is done through the means of developing and testing a candidate solution in the form of hybrid explanations, which essentially combine both visual and textual explanations. This hybrid form of explanations shows a significant improvement in terms of correct understanding (for lay users in particular) when compared to visual explanations, whilst not compromising on ease-of-use at the same time. CCS CONCEPTS • Human-centered computing →Empirical studies in visualiza- tion; User studies; User models; • Information systems →De- cision support systems; • Computing methodologies →Arti- ficial intelligence.

UX of AI & PersonalizationResearch Papers

Anchoring Bias Affects Mental Model Formation and User Reliance in Explainable AI SystemsMahsan Nourani · 2021EXplainable Artificial Intelligence (XAI) approaches are used to bring transparency to machine learning and artificial intelligence models, and hence, improve the decision-making process for their end-users. While these methods aim to improve human understand- ing and their mental models, cognitive biases can still influence a user’s mental model and decision-making in ways that system de- signers do not anticipate. This paper presents research on cognitive biases due to ordering effects in intelligent systems. We conducted a controlled user study to understand how the order of observing sys- tem weaknesses and strengths can affect the user’s mental model, task performance, and reliance on the intelligent system, and we investigate the role of explanations in addressing this bias. Using an explainable video activity recognition tool in the cooking domain, we asked participants to verify whether a set of kitchen policies are being followed, with each policy focusing on a weakness or a strength. We controlled the order of the policies and the presence of explanations to test our hypotheses. Our main finding shows that those who observed system strengths early-on were more prone to automation bias and made significantly more errors due to positive first impressions of the system, while they built a more accurate mental model of the system competencies. On the other hand, those who encountered weaknesses earlier made significantly fewer er- rors since they tended to rely more on themselves, while they also underestimated model competencies due to having a more negative first impression of the model. Our work presents strong findings that aim to make intelligent system designers aware of such biases when designing such tools. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. IUI ’21, April 14–17, 2021, College Station, TX, USA © 2021 Association for Computing Machinery. ACM ISBN 978-1-4503-8017-1/21/04...$15.00 https://doi.org/10.1145/3397481.34

UX of AI & PersonalizationResearch Papers

On-demand Personalized Explanation for Transparent RecommendationMouadh Guesmi · 2021The literature on explainable recommendations is already rich. In this paper, we aim to shed light on an aspect that remains under- explored in this area of research, namely providing personalized explanations. To address this gap, we developed a transparent Rec- ommendation and Interest Modeling Application (RIMA) that pro- vides on-demand personalized explanations with varying levels of detail to meet the demands of different types of end-users. The results of a preliminary qualitative user study demonstrated po- tential benefits in terms of user satisfaction with the explainable recommender system. Our work would contribute to the litera- ture on explainable recommendation by exploring the potential of on-demand personalized explanations, and contribute to the prac- tice by offering suggestions for the design and appropriate use of personalized explanation interfaces in recommender systems. CCS CONCEPTS • Information systems →Personalization; Recommender sys- tems.

UX of AI & PersonalizationResearch Papers

Progressive Disclosure: When, Why, and How Do Users Want Algorithmic Transparency Information?Arron Springer, Steve Whittaker · 2020It is essential that users understand how algorithmic decisions are made, as we increasingly delegate important decisions to intelligent systems. Prior work has often taken a techno-centric approach, focusing on new computational techniques to support transparency. In contrast, this article employs empirical methods to better understand user reactions to transparent systems to motivate user-centric designs for transparent systems. We assess user reactions to transparency feedback in four studies of an emotional analytics system. In Study 1, users anticipated that a transparent system would perform better but unexpectedly retracted this evaluation after experience with the system. Study 2 offers an explanation for this paradox by showing that the benefits of transparency are context dependent. On the one hand, transparency can help users form a model of the underlying algorithm's operation. On the other hand, positive accuracy perceptions may be undermined when transparency reveals algorithmic errors. Study 3 explored real-time reactions to transparency. Results confirmed Study 2, in showing that users are both more likely to consult transparency information and to experience greater system insights when formulating a model of system operation. Study 4 used qualitative methods to explore real-time user reactions to motivate transparency design principles. Results again suggest that users may benefit from initially simplified feedback that hides potential system errors and assists users in building working heuristics about system operation. We use these findings to motivate new progressive disclosure principles for transparency in intelligent systems and discuss theoretical implications.

UX of AI & PersonalizationResearch Papers

Exploring Mental Models for Transparent and Controllable Recommender Systems: A Qualitative StudyThao Ngo · 2020While online content is personalized to an increasing degree, e.g. us- ing recommender systems (RS), the rationale behind personalization and how users can adjust it typically remains opaque. This was often observed to have negative effects on the user experience and perceived quality of RS. As a result, research increasingly has taken user-centric aspects such as transparency and control of a RS into account, when assessing its quality. However, we argue that too little of this research has investigated the users’ perception and understanding of RS in their entirety. In this paper, we explore the users’ mental models of RS. More specifically, we followed the qualitative grounded theory methodology and conducted 10 semi- structured face-to-face interviews with typical and regular Netflix users. During interviews participants expressed high levels of un- certainty and confusion about the RS in Netflix. Consequently, we found a broad range of different mental models. Nevertheless, we also identified a general structure underlying all of these models, consisting of four steps: data acquisition, inference of user profile, comparison of user profiles or items, and generation of recommen- dations. Based on our findings, we discuss implications to design more transparent, controllable, and user friendly RS in the future. CCS CONCEPTS • Information systems →Recommender systems; • Human- centered computing →User studies.

UX of AI & PersonalizationResearch Papers

Questioning the AI: Informing Design Practices for Explainable AI User ExperincesLiao · 2020A surge of interest in explainable AI (XAI) has led to a vast collection of algorithmic work on the topic. While many rec- ognize the necessity to incorporate explainability features in AI systems, how to address real-world user needs for under- standing AI remains an open question. By interviewing 20 UX and design practitioners working on various AI products, we seek to identify gaps between the current XAI algorithmic work and practices to create explainable AI products. To do so, we develop an algorithm-informed XAI question bank in which user needs for explainability are represented as proto- typical questions users might ask about the AI, and use it as a study probe. Our work contributes insights into the design space of XAI, informs efforts to support design practices in this space, and identifies opportunities for future XAI work. We also provide an extended XAI question bank and discuss how it can be used for creating user-centered XAI. Author Keywords Explainable AI; human-AI interaction; User experience

UX of AI & PersonalizationResearch Papers

To Explain or not to Explain: the Effects of Personal Characteristics when Explaining Music RecommendationsMartijn Millecamp · 2019Recommender systems have been increasingly used in online services that we consume daily, such as Facebook, Netflix, YouTube, and Spotify. However, these systems are often pre- sented to users as a “black box”, i.e. the rational for providing individual recommendations remains unexplained to users. In recent years, various attempts have been made to address this black box issue by providing textual explanations or interac- tive visualisations that enable users to explore the provenance of recommendations, and benefits in terms of precision and user satisfaction, among others, have been demonstrated. Pre- vious research had also indicated that personal characteristics such as domain knowledge, trust propensity and persistence may also play an important role on such perceived benefits. Yet, to date, little is known about the effects of personal char- acteristics when explaining recommendations. To address this gap, we developed a music recommender system with expla- nations and conducted an online study using a within-subject design. We captured various personal characteristics of par- ticipants and administered both qualitative and quantitative evaluation methods. Results indicated that personal character- istics have some significant influence on the interaction and perception of recommender systems and that this influence changes by adding explanations. Especially people with a low need for cognition benefited from explained recommendations. For people with a high need for cognition, we observed that explanations could lower their confidence. Based on these re- sults, we present some first design implications for explaining recommendations. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. IUI ’19, © 2019 Copyright held by the owner/author(s). Publication rights licensed to ACM. ISBN 123-4567-24-567/08/06...$15.00 DOI: http://dx.doi.org/10.475/123_4 ACM Classification Keywords H.5.2 Information Interfaces and Presentation (e.g. HCI): User-cen

UX of AI & PersonalizationResearch Papers

Personalized Explanations for Hybrid Recommender SystemsPigi Kouki · 2019Recommender systems have become pervasive on the web, shaping the way users see information and thus the decisions they make. As these systems get more complex, there is a growing need for transparency. In this paper, we study the problem of generating and visualizing personalized explanations for hybrid recommender sys- tems, which incorporate many different data sources. We build upon a hybrid probabilistic graphical model and develop an approach to generate real-time recommendations along with personalized explanations. To study the benefits of explanations for hybrid rec- ommender systems, we conduct a crowd-sourced user study where our system generates personalized recommendations and explana- tions for real users of the last.fm music platform. We experiment with 1) different explanation styles (e.g., user-based, item-based), 2) manipulating the number of explanation styles presented, and 3) manipulating the presentation format (e.g., textual vs. visual). We apply a mixed model statistical analysis to consider user personal- ity traits as a control variable and demonstrate the usefulness of our approach in creating personalized hybrid explanations with different style, number, and format. CCS CONCEPTS • Information systems →Decision support systems; Collabo- rative filtering; • Human-centered computing →Social network- ing sites; Empirical studies in visualization.

UX of AI & PersonalizationResearch Papers

Explanations as Mechanisms for Supporting Algorithmic TransparencyEmilee Rader · 2018Transparency can empower users to make informed choices about how they use an algorithmic decision-making system and judge its potential consequences. However, transparency is often conceptualized by the outcomes it is intended to bring about, not the specifics of mechanisms to achieve those out- comes. We conducted an online experiment focusing on how different ways of explaining Facebook’s News Feed algorithm might affect participants’ beliefs and judgments about the News Feed. We found that all explanations caused participants to become more aware of how the system works, and helped them to determine whether the system is biased and if they can control what they see. The explanations were less effective for helping participants evaluate the correctness of the system’s output, and form opinions about how sensible and consistent its behavior is. We present implications for the design of trans- parency mechanisms in algorithmic decision-making systems based on these results. Author Keywords algorithmic decision-making; transparency; explanations ACM Classification Keywords H.5.m. Information Interfaces and Presentation (e.g. HCI): Miscellaneous

UX of AI & PersonalizationResearch Papers

Explanation in artificial intelligence: Insights from the social sciencesTim Miller · 2018Article history: Received 22 June 2017 Received in revised form 17 May 2018 Accepted 16 July 2018 Available online 27 October 2018

UX of AI & PersonalizationResearch Papers

Towards Algorithmic Experience: Initial Efforts for Social Media ContextsOscar Alvarado · 2018Algorithms influence most of our daily activities, decisions, and they guide our behaviors. It has been argued that algorithms even have a direct impact on democratic societies. Human - Computer Interaction research needs to develop analytical tools for describing the interaction with, and experience of algorithms. Based on user participatory workshops focused on scrutinizing Facebook’s newsfeed, an algorithm-influenced social media, we propose the concept of Algorithmic Experience (AX) as an analytic framing for making the interaction with and experience of algorithms explicit. Connecting it to design, we articulate five functional categories of AX that are particularly important to cater for in social media: profiling transparency and management, algorithmic awareness and control, and selective algorithmic memory. Author Keywords Algorithms; algorithmic experience; social media; user- centered design; research through design ACM Classification Keywords H.5.2. User Interfaces: User-centered design

UX of AI & PersonalizationResearch Papers

User Preferences for Hybrid ExplanationsPigi Kouki · 2017Hybrid recommender systems combine several different sources of information to generate recommendations. These systems demon- strate improved accuracy compared to single-source recommen- dation strategies. However, hybrid recommendation strategies are inherently more complex than those that use a single source of information, and thus the process of explaining recommendations to users becomes more challenging. In this paper we describe a hybrid recommender system built on a probabilistic programming language, and discuss the benefits and challenges of explaining its recommendations to users. We perform a mixed model statis- tical analysis of user preferences for explanations in this system. Through an online user survey, we evaluate explanations for hybrid algorithms in a variety of text and visual, graph-based formats, that are either novel designs or derived from existing hybrid recom- mender systems.

UX of AI & PersonalizationResearch Papers

“Algorithms ruin everything”: #RIPTwitter, Folk Theories, and Resistance to Algorithmic Change in Social MediaDarren Gergle · 2017As algorithmically-driven content curation has become an increasingly common feature of social media platforms, user resistance to algorithmic change has become more frequent and visible. These incidents of user backlash point to larger issues such as inaccurate understandings of how algorithmic systems work as well as mismatches between designer and user intent. Using a content analysis of 102,827 tweets from #RIPTwitter, a recent hashtag-based backlash to rumors about introducing algorithmic curation to Twitter’s timeline, this study addresses the nature of user resistance in the form of the complaints being expressed, folk theories of the algorithmic system espoused by users, and how these folk theories potentially frame user reactions. We find that resistance to algorithmic change largely revolves around expectation violation, with folk theories acting as frames for reactions such that more detailed folk theories are expressed through more specific reactions to algorithmic change. Author Keywords Algorithms; algorithm awareness; folk theories; technology continuance; user resistance; social media; algorithmic curation; expectation violation; machine classification ACM Classification Keywords H.5.m. Information interfaces and presentation (e.g., HCI):

UX of AI & PersonalizationResearch Papers

How Much Information? Effects of Transparency on Trust in an Algorithmic InterfaceRene Kizilcec · 2016The rising prevalence of algorithmic interfaces, such as cu- rated feeds in online news, raises new questions for designers, scholars, and critics of media. This work focuses on how trans- parent design of algorithmic interfaces can promote awareness and foster trust. A two-stage process of how transparency affects trust was hypothesized drawing on theories of infor- mation processing and procedural justice. In an online field experiment, three levels of system transparency were tested in the high-stakes context of peer assessment. Individuals whose expectations were violated (by receiving a lower grade than expected) trusted the system less, unless the grading algorithm was made more transparent through explanation. However, providing too much information eroded this trust. Attitudes of individuals whose expectations were met did not vary with transparency. Results are discussed in terms of a dual process model of attitude change and the depth of justification of per- ceived inconsistency. Designing for trust requires balanced interface transparency—not too little and not too much. ACM Classification Keywords H.5.2. Information Interfaces and Presentation (e.g. HCI): User Interfaces; K.3.1. Computers and Education: Computer Uses in Education. Author Keywords Interface Design; Algorithm Awareness; Attitude Change; Transparency; Trust; Peer Assessment.

UX of AI & PersonalizationResearch Papers

Crowd-Based Personalized Natural Language Explanations for RecommendationsShuo Chang · 2016Explanations are important for users to make decisions on whether to take recommendations. However, algorithm gen- erated explanations can be overly simplistic and unconvinc- ing. We believe that humans can overcome these limita- tions. Inspired by how people explain word-of-mouth rec- ommendations, we designed a process, combining crowd- sourcing and computation, that generates personalized nat- ural language explanations. We modeled key topical as- pects of movies, asked crowdworkers to write explanations based on quotes from online movie reviews, and personal- ized the explanations presented to users based on their rat- ing history. We evaluated the explanations by surveying 220 MovieLens users, finding that compared to personalized tag- based explanations, natural language explanations: 1) con- tain a more appropriate amount of information, 2) earn more trust from users, and 3) make users more satisfied. This paper contributes to the research literature by describing a scalable process for generating high quality and personalized natural language explanations, improving on state-of-the-art content-based explanations, and showing the feasibility and advantages of approaches that combine human wisdom with algorithmic processes.

UX of AI & PersonalizationResearch Papers