Somayeh Molaei
Advances in language modeling have paved the way for novel human-AI co-writing experiences. This paper explores how vary ing levels of scafolding from large language models (LLMs) shape the co-writing process. Employing a within-subjects feld experi ment with a Latin square design, we asked participants (N=131) to respond to argumentative writing prompts under three randomly sequenced conditions: no AI assistance (control), next-sentence sug gestions (low scafolding), and next-paragraph suggestions (high scafolding). Our fndings reveal a U-shaped impact of scafold ing on writing quality and productivity (words/time). While low scafolding did not signifcantly improve writing quality or productivity, high scafolding led to signifcant improvements, es pecially benefting non-regular writers and less tech-savvy users. No signifcant cognitive burden was observed while using the scaf folded writing tools, but a moderate decrease in text ownership and satisfaction was noted. Our results have broad implications for the design of AI-powered writing tools, including the need for personalized scafolding mechanisms. CCS CONCEPTS • Human-centered computing → Collaborative and social computing design and evaluation methods; Empirical studies in collaborative and social computing.