Antti Oulasvirt
| The graphical user interface (GUI) has become the prime means for interacting with computing systems. It lever- ages human perceptual and motor capabilities for elementary tasks such as command exploration and invocation, informa- tion search, and multitasking. For designing a GUI, numerous interconnected decisions must be made such that the out- come strikes a balance between human factors and technical objectives. Normally, design choices are specified manually and coded within the software by professional designers and developers. This article surveys combinatorial optimization as a flexible and powerful tool for computational generation and adaptation of GUIs. As recently as 15 years ago, applications were limited to keyboards and widget layouts. The obstacle has been the mathematical definition of design tasks, on the one hand, and the lack of objective functions that capture essential aspects of human behavior, on the other. This article presents definitions of layout design problems as integer programming tasks, a coherent formalism that permits identification of problem types, analysis of their complexity, and exploitation of known algorithmic solutions. It then surveys advances in Manuscript received April 8, 2019; revised October 9, 2019 and January 12, 2020; accepted January 17, 2020. This work was supported in part by the European Research Council (ERC) through the European Union’s Horizon 2020 Research and Innovation Program under Grant 637991 and in part by the Academy of Finland projects Bayesian Artefact Design (BAD) and Human Automata. (Corresponding author: Antti Oulasvirta.) Antti Oulasvirta is with the Department of Communications and Networking, School of Electrical Engineering, Aalto University, 02150 Espoo, Finland, and also with the Finnish Center for Artificial Intelligence (FCAI), 02015 Espoo, Finland (e-mail: antti.oulasvirta@aalto.fi). Niraj Ramesh Dayama and Morteza Shiripour are with the Department of Communications and Networking, School of Electrical Engineering, Aalto University, 02150 Espoo, Finland. Maximilian John and Andreas Karrenbauer are with the Max Planck Institute for Informatics, 66123 Saarbrücken, Germany. Digital Object Identifier 10.1109/JPROC.2020.2969687 formulating evaluative functions for common design-goal foci such as user performance and experience. The convergence of these two advances has expanded the range of solvable problems. Approaches to practical deployment are outlined with a wide spectrum of applica