Filtered by themeLarge language models8 papersclear ✕

Improving Language Understanding by Generative Pre-TrainingAlec RadforNatural language understanding comprises a wide range of diverse tasks such as textual entailment, question answering, semantic similarity assessment, and document classification. Although large unlabeled text corpora are abundant, labeled data for learning these specific tasks is scarce, making it challenging for discriminatively trained models to perform adequately. We demonstrate that large gains on these tasks can be realized by generative pre-training of a language model on a diverse corpus of unlabeled text, followed by discriminative fine-tuning on each specific task. In contrast to previous approaches, we make use of task-aware input transformations during fine-tuning to achieve effective transfer while requiring minimal changes to the model architecture. We demonstrate the effectiveness of our approach on a wide range of benchmarks for natural language understanding. Our general task-agnostic model outperforms discriminatively trained models that use architectures specifically crafted for each task, significantly improving upon the state of the art in 9 out of the 12 tasks studied. For instance, we achieve absolute improvements of 8.9% on commonsense reasoning (Stories Cloze Test), 5.7% on question answering (RACE), and 1.5% on textual entailment (MultiNLI).

UX of AI & PersonalizationResearch Papers

Analyzing behavioral intentions toward Generative Artificial Intelligence: the case of ChatGPTDongyan Nan · 2024Generative artificial intelligence (AI) is an innovative AI technology that has garnered considerable attention worldwide. This study aimed to facilitate the development of such technologies by examining the factors affecting individuals’ inten­ tions toward generative AI (e.g., ChatGPT). Concretely, we developed a causal model by extending the expectation confirmation model with information system success theory, privacy concerns, and perceived innovativeness. Then, we tested the model by analyzing survey-based data from 252 Korean ChatGPT users. As a result, we found that anteced­ ent variables -information quality, system quality, privacy concerns, and perceived innovativeness- play notable roles in affecting users’ intentions to continually use and recommend generative AI ChatGPT. Overall, the current research is one of the first attempts to track the variables influencing individuals’ intentions to continually use and recommend in the context of generative AI ChatGPT.

UX of AI & PersonalizationResearch Papers

Shaping Human-AI Collaboration: Varied Scafolding Levels in Co-writing with Language ModelsSomayeh Molaei · 2024Advances in language modeling have paved the way for novel human-AI co-writing experiences. This paper explores how vary­ ing levels of scafolding from large language models (LLMs) shape the co-writing process. Employing a within-subjects feld experi­ ment with a Latin square design, we asked participants (N=131) to respond to argumentative writing prompts under three randomly sequenced conditions: no AI assistance (control), next-sentence sug­ gestions (low scafolding), and next-paragraph suggestions (high scafolding). Our fndings reveal a U-shaped impact of scafold­ ing on writing quality and productivity (words/time). While low scafolding did not signifcantly improve writing quality or productivity, high scafolding led to signifcant improvements, es­ pecially benefting non-regular writers and less tech-savvy users. No signifcant cognitive burden was observed while using the scaf­ folded writing tools, but a moderate decrease in text ownership and satisfaction was noted. Our results have broad implications for the design of AI-powered writing tools, including the need for personalized scafolding mechanisms. CCS CONCEPTS • Human-centered computing → Collaborative and social computing design and evaluation methods; Empirical studies in collaborative and social computing.

UX of AI & PersonalizationResearch Papers

SituationAdapt: Contextual UI Optimization in Mixed Reality with Situation Awareness via LLM ReasoningZhipeng Li, Christoph Gebhardt et al. · 2024Mixed Reality is increasingly used in mobile settings beyond con­ trolled home and ofce spaces. This mobility introduces the need for user interface layouts that adapt to varying contexts. However, existing adaptive systems are designed only for static environments. In this paper, we introduce SituationAdapt, a system that adjusts Mixed Reality UIs to real-world surroundings by considering envi­ ronmental and social cues in shared settings. Our system consists of perception, reasoning, and optimization modules for UI adaptation. Our perception module identifes objects and individuals around the user, while our reasoning module leverages a Vision-and-Language Model to assess the placement of interactive UI elements. This en­ sures that adapted layouts do not obstruct relevant environmental cues or interfere with social norms. Our optimization module then generates Mixed Reality interfaces that account for these consid­ erations as well as temporal constraints. For evaluation, we frst validate our reasoning module’s capability of assessing UI contexts in comparison to human expert users. In an online user study, we then establish SituationAdapt’s capability of producing context- aware layouts for Mixed Reality, where it outperformed previous adaptive layout methods. We conclude with a series of applications and scenarios to demonstrate SituationAdapt’s versatility. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for proft or commercial advantage and that copies bear this notice and the full citation on the frst page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specifc permission and/or a fee. Request permissions from permissions@acm.org. UIST ’24, October 13–16, 2024, Pittsburgh, PA, USA © 2024 Copyright held by the owner/author(s). Publication rights licensed to ACM. ACM ISBN 979-8-4007-0628-8/24/10 https://doi.org/10.1145/3654777.3676470 CCS CONCEPTS • Human-centered computing → Mixed / augmented reality; Virtual reality; Interactive systems and tools.

UX of AI & PersonalizationResearch Papers

Predicting the usability of mobile applications using AI tools: the rise of large user interface models, opportunities, and challengesIndustry (EDI40) · 2024This article proposes the so-called large user interface models (LUIMs) to enable the generation of user interfaces and prediction of usability using artificial intelligence in the context of mobile applications. To this end, we synergized an integrated framework for the effective testing of the usability of mobile applications following a selective review of the most influential models of mobile usability testing. Next, we identified and analysed 13 recent AI tools that generate user interfaces for mobile apps, and systematically tested these tools to identify their AI capabilities. Our striking findings demonstrate that current generative UI tools fail to address mobile usability attributes, such as efficiency, learnability, effectiveness, satisfaction, and memorability. Our large UI models’ architecture proposes to leverage the capabilities of large language models, large vision models, and large code models to overcome the challenges of AI-driven UI/UX design and front-end implementations. This fascinating UI eco-system must be augmented with sufficient UI data and multi-sensory input regarding user behaviour to train the models. We anticipate LUIMs to create ample opportunities, like expedited frontend software development, enhanced personalised user experience, and wider accessibility of smart technologies. However, the research challenges hindering the UI generation and usability prediction of mobile apps include the seamless integration of complex generative AI models, semantic understanding of non-uniform visual designs, scarcity of UX datasets, and modelling of realistic user interactions. 672 Abdallah Namoun et al. / Procedia Computer Science 238 (2024) 671–682

UX of AI & PersonalizationResearch Papers

CulturAI: Exploring Mixed Reality Art Exhibitions with Large Language Models for Personalized Immersive ExperiencesNicolas Constantinides · 2024Mixed Reality (MR) technologies have transformed the way in which we interact and engage with digital content, offering immer- sive experiences that blend the physical and virtual worlds. Over the past years, there has been increasing interest in employing Artificial Intelligence (AI) technologies to improve user experience and trustworthiness in cultural contexts. However, the integration of Large Language Models (LLMs) into MR applications within the Cultural Heritage (CH) domain is relatively underexplored. In this work, we present an investigation into the integration of LLMs within MR environments, focusing on the context of virtual art exhi- bitions. We implemented a HoloLens MR application, which enables users to explore artworks while interacting with an LLM through voice. To evaluate the user experience and perceived trustworthi- ness of individuals engaging with an LLM-based virtual art guide, we adopted a between-subject study design, in which participants were randomly assigned to either the LLM-based version or a con- trol group using conventional interaction methods. The LLM-based version allows users to pose inquiries about the artwork displayed, ranging from details about the creator to information about the artwork’s origin and historical significance. This paper presents the technical aspects of integrating LLMs within MR applications and evaluates the user experience and perceived trustworthiness of this approach in enhancing the exploration of virtual art exhibitions. Results of an initial evaluation provide evidence about the positive aspect of integrating LLMs in MR applications. Findings of this work contribute to the advancement of MR technologies for the development of future interactive personalized art experiences. CCS CONCEPTS • Human-centered computing →Mixed / augmented reality; User studies; • Computing methodologies →Artificial intelligence. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s). UMAP Adjunct ’24, July 01–04, 2024, Cagliari, Italy © 2024 Copyright held by the owner/author(s). ACM ISBN 979-8-4007-0466-6/24/07 https://doi.org/10.1145/3631700.3664874

UX of AI & PersonalizationResearch Papers

Large Language Models Meet User Interfaces: The Case of Provisioning FeedbackStanislav Pozdniakov, Jonathan Brazil et al. · 2024- a *School of Electrical Engineering and Computer Science, The University of Queensland, St Lucia, QLD, 4072, Australia* - b *Institute for Teaching and Learning Innovation, The University of Queensland, St Lucia, QLD, 4072, Australia* - c *Centre for Learning Analytics, Faculty of Information Technology, Monash University, Melbourne, VIC, 3800, Australia* - d *School of Computer Science, University of Auckland, 38 Princes Street Auckland, 1010, New Zealand*

UX of AI & PersonalizationResearch Papers

Farsighted-Fostering Responsable AI awarness during AI application prototypingWang · 2024Prompt-based interfaces for Large Language Models (LLMs) have made prototyping and building AI-powered applications easier than ever before. However, identifying potential harms that may arise from AI applications remains a challenge, particularly during ∗The work was done when the authors were at Google Research. This work is licensed under a Creative Commons Attribution 4.0 International License. CHI ’24, May 11–16, 2024, Honolulu, HI, USA © 2024 Copyright held by the owner/author(s). ACM ISBN 979-8-4007-0330-0/24/05. [https://doi.org/10.1145/3613904.3642335](https://doi.org/10.1145/3613904.3642335) prompt-based prototyping. To address this, we present Farsight, a novel in situ interactive tool that helps people identify potential harms from the AI applications they are prototyping. Based on a user’s prompt, Farsight highlights news articles about relevant AI incidents and allows users to explore and edit LLM-generated use cases, stakeholders, and harms. We report design insights from a co- design study with 10 AI prototypers and findings from a user study with 42 AI prototypers. After using Farsight, AI prototypers in our user study are better able to independently identify potential harms associated with a prompt and find our tool more useful and usable than existing resources. Their qualitative feedback also highlights that Farsight encourages them to focus on end-users and think beyond immediate harms. We discuss these findings and reflect on 1 arXiv:2402.15350v2 [cs.HC] 2 Jul 2024 CHI ’24, May 11–16, 2024, Honolulu, HI, USA Zijie J. Wang, et al. their implications for designing AI prototyping experiences that meaningfully engage with AI harms. Farsight is publicly accessible at: [https://pair-code.github.io/farsight](https://pair-code.github.io/farsight). CCS CONCEPTS • Human-centered computing →Interactive systems and tools; • Computing methodologies →Machine learning.

UX of AI & PersonalizationResearch Papers