Fumeng Yang
We investigated the effects of example-based explanations for a machine learning classifier on end users’ appropriate trust. We explored the effects of spatial layout and visual representation in an in-person user study with 33 participants. We measured partici- pants’ appropriate trust in the classifier, quantified the effects of different spatial layouts and visual representations, and observed changes in users’ trust over time. The results show that each expla- nation improved users’ trust in the classifier, and the combination of explanation, human, and classification algorithm yielded much better decisions than the human and classification algorithm sepa- rately. Yet these visual explanations lead to different levels of trust and may cause inappropriate trust if an explanation is difficult to un- ∗Fumeng Yang was a PhD intern at Pacific Northwest National Laboratory when conducting this research. †Jean Scholtz retired from Pacific Northwest National Laboratory September 2018. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s). IUI ’20, March 17–20, 2020, Cagliari, Italy © 2020 Copyright held by the owner/author(s). ACM ISBN 978-1-4503-7118-6/20/03. [https://doi.org/10.1145/3377325.3377480](https://doi.org/10.1145/3377325.3377480) derstand. Visual representation and performance feedback strongly affect users’ trust, and spatial layout shows a moderate effect. Our results do not support that individual differences (e.g., propensity to trust) affect users’ trust in the classifier. This work advances the state-of-the-art in trust-able machine learning and informs the design and appropriate use of automated systems. CCS CONCEPTS • Human-centered computing →Empirical studies in HCI; Information visualization; Empirical studies in visualization; Visualization design and evaluation methods; • Computing method- ologies →Supervised learning by classification.