Phong Minh Vu, Hung Viet Pham, Tung Thanh Nguyen
User interface (UI) is one of the most important components of a mobile app and strongly influences users’ perception of the app. However, UI design tasks are typically manual and time-consuming. This paper proposes a novel approach to (semi)-automate those tasks. Our key idea is to develop and deploy advanced deep learning models based on recurrent neural networks (RNN) and generative adversarial networks (GAN) to learn UI design patterns from mil- lions of currently available mobile apps. Once trained, those models can be used to search for UI design samples given user-provided descriptions written in natural language and generate professional- looking UI designs from simpler, less elegant design drafts.