Looks Fine to Me: Understanding Oversight in Agentic Interactions
No description
https://arxiv.org/html/2610.27801v1Reader - page text saved at the timeLooks Fine to Me: Understanding Oversight in Agentic Interactions
CCS: Human-centered computing Empirical studies in HCI
CCS: Computing methodologies Intelligent agents
Dinithi Dissanayake
Augmented Human Lab
National University of Singapore, Singapore, Singapore
dinithid@u.nus.edu
Yasith Samaradivakara
MIT Media Lab
MIT, Cambridge, Massachusetts, USA
yasith@media.mit.edu
Hyung Woon Lee
MIT Media Lab
MIT, Cambridge, Massachusetts, USA
hyung98@media.mit.edu
Yadeesha Weerasinghe
Augmented Human Lab
National University of Singapore, Singapore, Singapore
yadeesha@ahlab.org
Sanjay Saha
Augmented Human Lab
National University of Singapore, Singapore, Singapore
sanjaysaha@nus.edu.sg
Pattie Maes
MIT Media Lab
MIT, Cambridge, Massachusetts, USA
pattie@media.mit.edu
Suranga Nanayakkara
Augmented Human Lab
National University of Singapore, Singapore, Singapore
suranga@ahlab.org
Received 5 June 2009
Figure 1.Participants completed Python programming tasks with a coding agent while we collected egocentric sensing (FPV, gaze), physiological sensing (PPG, HRV), and interface-interaction logs (cursor movements, keyboard clicks, interface and agent interactions). Across tasks, the agent produced correct code (ND), a silent defect that passed the hidden pre-merge checks (SD), or a visible failure that caused the checks to fail (BD). We examined how participants progressed from surface-level checking to task-relevant evaluation, defect detection, and successful resolution.
Abstract.
Agentic AI systems are deployed to execute complex tasks across domains such as writing, coding, and data analysis. As execution is offloaded to AI agents, the human role shifts toward oversight. Effective oversight requires recognizing when closer scrutiny is needed, even though flawed outputs may appear convincing. Our formative study (
𝑁
=
12
) revealed that users accepted plausible outputs without thorough evaluation, and only recognized these lapses in retrospect. To examine this systematically, we conducted a controlled within-subject study (
𝑁
=
36
) in which participants completed programming tasks with an agent that produced correct outputs, silent defects, or visible failures. Combining task outcomes, interaction logs, gaze, physiological signals, and self-reports, we examined perceived, behavioral, and effective oversight. Participants were often highly confident while missing silent defects: they performed generic checks, failed to generate diagnostic evidence, or even misinterpreted defect evidence. We discuss how studying oversight across multiple dimensions can support future work on timely, task-calibrated re-engagement.
1.Introduction
Generative AI is changing not only what people can do, but also how they are involved in the process. With a single prompt, AI agents can now interpret goals, plan and execute actions, modify artifacts directly, and return completed work. As users continue to offload task execution to AI agents, user roles are shifting towards oversight. However, ensuring that completed work aligns with user intent requires oversight that is active, not passive; users need to trace and understand the agent’s actions, and evaluate if the agent has performed their task correctly.
Prior works discuss the necessity of remaining cognitively engaged, as past successes cannot guarantee future ones (Chen et al., 2026). Repeated acceptance of outputs without thorough evaluation reduces opportunities for learning and critical thinking, while leaving users less prepared to catch subtle errors and failures, or intervene when an agent’s actions diverge from their intent (Lee et al., 2025; Kazemitabaar et al., 2025). However, as agents become increasingly capable, checking every action becomes unnecessary and costly. Moreover, with repeated successes, it becomes harder for users to constantly remain cognitively engaged, and users may gradually fall out of the loop (Endsley and Kiris, 1995; Parasuraman et al., 1993; Bainbridge, 1983; McCabe et al., 2024). We therefore employ the term effective oversight to mean recognizing when closer evaluation is needed and re-engaging at the right moment. Conversely, a lapse in effective oversight would describe the phenomenon where users remain involved in an agentic workflow and accept outputs while insufficiently evaluating the substance of the agent’s work.
Effective oversight is often a challenge for users, given that flawed outputs are often generated as complete and plausible responses. Langer describes a similar failure to spot cues while overlooking differences that matter in the current situation (Langer et al., 1978; Langer, 1989). This characterization of effective oversight raises questions on whether users recognize lapses in their oversight, how these lapses manifest during interaction, and what prompts them to re-engage. To investigate oversight in agentic interaction, we first conducted a formative study (
𝑁
=
12
) on everyday AI use and discovered that lapses in effective oversight within agentic interactions often occur without the user even being aware of it. Guided by our findings, we identified AI-assisted programming as a prominent context to study oversight, and subsequently conducted a controlled within-subject study with 36 participants completing programming tasks with an AI coding agent. The agent either produced a correct solution, introduced a plausible defect without an external failure signal, or introduced a defect surfaced by a visible failure. Our findings show that participants often missed silent defects despite remaining confident, but readily identified faulty outputs when failures were visible, suggesting that visible failure cues played an important role in prompting effective oversight. Overall, this paper makes two contributions:
•
Insights on how people experienced reduced oversight with AI. Our formative study (
𝑁
=
12
) shows that users can describe reduced oversight but may not recognize it as it happens. We share several themes spanning across interaction behaviour, attention, and bodily state, as well as explore and chart the directions this exploration opens.
•
Findings on ‘oversight’ under repeated agent interactions based on multimodal data analysis. Through a controlled study (
𝑁
=
36
) combining task outcomes, interaction logs, gaze, and physiological signals, we examine how oversight changes between interactions with correct, silently flawed, and visibly failing agent outputs, and how these patterns vary with user programming experience. We reveal differences between feeling vigilant and being vigilant and discuss their implications for measuring oversight and supporting timely re-engagement.
2.Related Work
2.1.Cognitive Engagement and Oversight in Human–AI Interaction
Concerns about maintaining human cognitive engagement have long accompanied increasing levels of automation. Bainbridge’s “ironies of automation” highlighted how automating routine execution can shift people toward supervisory roles while reducing the experience needed to intervene effectively when automation fails (Bainbridge, 1983). Subsequent work showed that reliable automation can foster complacency and reduced monitoring, while higher levels of automation can leave users “out of the loop” and less able to understand system state or intervene effectively (Parasuraman et al., 1993; Endsley and Kiris, 1995; Parasuraman and Riley, 1997). Related work on automation bias and reliance further shows that people may defer to automated recommendations rather than independently seek or evaluate information, with reliance shaped by factors such as trust, workload, and system reliability (Skitka et al., 2000; Dzindolet et al., 2003; Lee and See, 2004; Parasuraman and Manzey, 2010). As systems take over more of the execution, the human role increasingly shifts toward oversight. Yet sustained attention and evaluation can become more difficult as continuous involvement in the task decreases.
Generative AI extends this shift from supervising automated processes to evaluating AI-generated work. As users delegate more cognitive work to AI, studies have raised concerns about cognitive offloading and reduced critical-thinking effort (Gerlich, 2025; Lee et al., 2025). Delegation does not necessarily remove the need for human evaluation. Users must still decide what to delegate, assess generated outputs, and determine when further scrutiny is needed (Tankelevitch et al., 2024). When such evaluation is insufficient, users may over-rely on AI and accept incorrect recommendations (Buçinca et al., 2021; de Jong et al., 2025). This has motivated interventions such as cognitive forcing functions, which encourage users to engage more deliberately with a problem before or while receiving AI assistance (Buçinca et al., 2021; de Jong et al., 2025).
However, much of this work characterizes cognitive engagement through its outcomes, such as reliance on incorrect advice or task performance (Buçinca et al., 2021; de Jong et al., 2025), through users’ retrospective reports of critical-thinking effort (Lee et al., 2025), or by evaluating interventions designed to increase deliberation (Buçinca et al., 2021; de Jong et al., 2025). Less is understood about how the quality of oversight changes during ongoing interaction, particularly when users remain behaviorally involved with an AI system and nothing explicitly signals that greater scrutiny is necessary. Human involvement alone does not necessarily imply effective oversight (Zhu et al., 2026): a user may continue prompting, reviewing, or approving an agent’s actions while attending only superficially to details that require human judgment. We examine this in repeated interactions with increasingly autonomous AI agents.
2.2.From AI-Assisted Coding to Agentic Programming
AI-assisted programming has evolved from supporting developers during coding to increasingly delegating programming tasks to AI systems. Code-completion tools operate within developer-led workflows, leaving programmers to inspect, accept, modify, or reject generated suggestions (Mozannar et al., 2024). More recent coding agents can instead take higher-level goals, plan changes, modify files, execute code and tests, and iterate toward solutions with substantially less developer involvement (Takerngsaksiri et al., 2025; Chen et al., 2026). This progression represents not simply an increase in generation capability, but a redistribution of work between developers and AI systems.
Studies of agentic development workflows show that agents can take substantial responsibility for execution while humans retain responsibility for reviewing, steering, and accepting their work (Chung and Hassan, 2026; Wang et al., 2026; Dhanorkar et al., 2026). Yet verification can itself be demanding. Developers must evaluate implementations they may not have directly produced or closely followed (McCabe et al., 2024), and repeated verification can impose cognitive load and contribute to fatigue (Fan et al., 2026). Prior work has therefore examined how developers critically engage with generated code and how deeper engagement might be supported (Hutchison et al., 2026; Kazemitabaar et al., 2025). At the same time, studies of AI-assisted programming raise concerns that increased generation may not be accompanied by equally rigorous validation (O’Brien and others, 2025). As coding agents become more autonomous, effective oversight increasingly depends on how developers evaluate work they did not directly produce.
This challenge is particularly consequential when an agent appears to have completed a task successfully. Real-world coding-agent interactions reveal failures arising from mismatches between what developers intend and what agents interpret, implement, or report (Tang et al., 2026). Moreover, developers can have difficulty distinguishing correct from incorrect AI-generated artifacts even while remaining confident in their judgments (Kaufman and others, 2026). Some failures provide clear cues for further investigation, such as failed tests, execution errors, or visibly incorrect results. When an implementation appears plausible and available checks pass, however, users must independently determine whether the evidence of success is sufficient. Less is understood about whether developers recognize when such apparent success warrants deeper scrutiny and whether their evaluation can reveal errors that are not externally signaled.
2.3.Modeling Cognitive Engagement from Behavior
Understanding the quality of human oversight requires looking beyond whether users remain active in an interaction. Prior work suggests that differences in cognitive engagement can be reflected in how people behave, allocate attention, and respond physiologically while performing a task (Li, 2021; D’Mello and Graesser, 2010; Kosch et al., 2023). Behavioral measures such as pauses, response timing, and action sequences can reflect changes in how users approach a task (D’Mello and Graesser, 2009; Stiber et al., 2024); gaze measures provide evidence about the allocation of visual attention; and physiological measures have been studied as indicators of workload, arousal, and mental effort (Tao et al., 2019; Charles and Nixon, 2019; Kosch et al., 2023). These signals offer potential evidence about cognitive engagement, but their meaning is rarely unambiguous. The same behavior or physiological response may reflect different cognitive processes depending on the task and moment in which it occurs. Cognitive engagement is therefore better characterized by considering multiple signals and the context in which they occur, rather than interpreting individual measures in isolation (Li, 2021).
This need for context is particularly important in Human–AI interaction, where users’ actions can provide task-grounded evidence of how they engage with AI-generated outputs. Interaction traces such as prompting, accepting or rejecting suggestions, editing, testing, and sequences of actions have been used to characterize how developers interact with and evaluate generated code (Mozannar et al., 2024; Hutchison et al., 2026). More broadly, sequences of programming actions can reveal strategies that are difficult to infer from individual low-level behaviors alone (Fern et al., 2007). For oversight, however, observable activity does not necessarily imply substantive evaluation. Continued interaction may reflect careful inspection, but it may also consist of routine actions that provide little evidence that the agent’s work has been critically examined. Conversely, periods of limited interaction may involve careful reading or reasoning. Thus, interpreting behavioral traces as evidence of oversight requires considering not only whether users interact, but what they are doing and how those actions relate to the task and the agent’s work.
Self-report provides a complementary perspective by capturing users’ perceived effort, workload, confidence, and engagement (Greene, 2015; Li, 2021). Yet subjective, behavioral, physiological, and performance-based measures need not converge (Charles and Nixon, 2019; Kosch et al., 2023). This distinction is particularly important for human oversight of AI agents: users may perceive themselves as attentive and exhibit behaviors consistent with evaluation without actually scrutinizing the aspects of an agent’s work that require human judgment. Effective oversight therefore cannot be inferred from perceived or observable engagement alone. Understanding oversight requires considering these indicators alongside whether users perform task-relevant evaluation capable of revealing problems in the agent’s work. This motivates examining oversight across perceived, behavioral, and effective dimensions, and asking when these dimensions align—and when they diverge.
3.Formative Study
We conducted a formative interview study to understand how people experience changes in their oversight during everyday AI use. We aimed to identify when users reduce oversight, whether they recognize this change, and how it might manifest in observable behavior. This formative study provided an empirical basis for selecting the context, manipulations, and measurements used in our controlled study.
3.1.Participants & Procedure
We conducted semi-structured interviews with 12 regular users of generative AI
(
(
𝑀
age
=
27.7
)
CLOSE
; 5 Females and 7 Males). Participants included four undergraduate students (F2, F3, F7, F10), four graduate students (F5, F6, F11, F12), and four working professionals ([F1, F4, F8, F9]). Participants used generative AI for activities including programming, writing, learning, and information seeking. Participants first described how they used AI in their everyday work. We then asked them to recall situations in which they had accepted or acted on AI output without carefully evaluating it. Follow-up questions examined what contributed to these situations, whether participants recognized them as they occurred, and what changes they noticed in their interaction, attention, or bodily state. Participants also reflected on the benefits and risks of reduced engagement and whether an external cue might help them re-engage. The full interview protocol is provided in the Appendix Section A.
We used a primarily deductive coding approach, with broad analytic categories derived from the interview guide. We coded excerpts describing participants’ AI-use practices, experiences of reduced evaluation, perceived triggers and consequences, observable signs, and awareness of these changes. The research team regularly discussed interviews from the initial participants to refine the interview guide, identify recurring codes, and clarify code definitions. The first author then coded the full dataset, while a second author reviewed the resulting coding. We compared related codes across participants and organized them into higher-level patterns. Below, we describe these patterns and how they informed the design of our subsequent controlled study.
3.2.Findings
Oversight was calibrated to its perceived value
Participants did not consider close evaluation necessary in every interaction. Instead, they adjusted their oversight according to the perceived stakes, available time, interest in the task, and confidence in the AI. Under time pressure, participants prioritized completing the task over understanding how the output had been produced. F5 explained, “When I am rushing, I am more mindless because I just need to get it done, even if it is lower quality." Similarly, low-stakes tasks gave participants little reason to invest in verification. As F8 described, “If the task is high-priority or high-value …I will not directly use the output the AI gives me," whereas low-stakes outputs were more readily accepted.
Repeated success also reduced the perceived value of continued checking. F1 described initially examining the system’s work but later deciding, “Once I know it is doing a good job, then I become mindless. I think, Okay, I can offload this. Let it do it." Thus, reduced oversight was not always an indiscriminate tendency to trust AI. It was often a contextual response to the apparent costs and benefits of further evaluation.
Participants also recognized benefits to this offloading. It saved time, reduced effort, and allowed them to focus their attention elsewhere. However, they described a longer-term tradeoff involving learning and skill. F11 explained, “If we do not use AI tools, we will be behind, because others are using AI tools …It is a tradeoff." F12 similarly felt that AI made work faster but that “my coding skills are getting rusty because I generate everything from AI." These accounts distinguish efficient delegation from the loss of evaluation needed to learn from, take responsibility for, or intervene in the work.
Procedural interaction did not guarantee evaluation
Participants described remaining active in an AI interaction without necessarily reasoning about its content. They could continue prompting, approve actions, or repeatedly ask the AI to fix a problem while focusing only on whether the final result worked. F3 explained, “You know what the end goal is, and you know what the final result should be, but you do not care what commands or outputs the AI is giving as long as the end goal is achieved." The same participant emphasized that even a long interaction could remain mindless when follow-ups consisted mainly of responses such as "yes," "no," or "this did not work, fix this."
Agentic tools appeared to strengthen this separation between participation and evaluation. F3 contrasted earlier copy-and-paste interactions with coding agents that iterate independently: “Now, with Codex and similar tools, you ask it one shot and it does it. You do not have to go back and forth because they iterate on their own …you do not think." Participants commonly evaluated only the visible result. F2 described asking an agent to implement something, checking a few functions, and continuing without reading the generated code. F11 similarly stated, “I do not go through much of what it is doing unless it is not working."
These accounts suggest that the amount of interaction is not, by itself, evidence of effective oversight. Users may remain procedurally involved while attending only to surface indicators such as whether the agent has finished, whether the output appears plausible, or whether the application runs.
Awareness ranged from deliberate offloading to retrospective recognition
Participants differed in whether they recognized their reduced engagement. Some intentionally chose not to evaluate low-priority work. F3 stated, “I know I am doing it. I know it is not good …I do not think it is an awareness problem. It is more of a discipline problem." F5 similarly described remaining in control and intentionally becoming less engaged when the task did not warrant greater effort.
Other accounts indicated that reduced oversight could occur without being noticed at the time. F12 explained, "If I am really busy, I might not even realize that I am doing it mindlessly." Several participants described recognizing the problem only when an output violated their expectations. F6 recalled that after an AI cited an article that did not exist, the failure "snapped me out of it" and served as a reminder "to put in more effort, verify things, and check things."
These findings complicate any simple claim that users are either aware or unaware of lapses in effective oversight. Awareness was contextual: users could deliberately offload evaluation in one situation but fail to notice inadequate oversight in another. Importantly, a visible failure could expose this mismatch and reactivate scrutiny. Whether that re-engagement persists beyond the immediate failure, however, remained unclear.
Reduced oversight had diverse observable signs
Participants associated reduced oversight with changes in how they prompted, read, verified, and distributed their attention. Short or underspecified prompts were mentioned frequently. F1 explained, “If I am thinking about it, my prompt is long …If I am mindless, I am more like, ‘Can you do this?’" Participants also described fewer substantive follow-ups, limited external verification, skimming summaries, and approving commands without examining them.
Attention often shifted away while the agent worked. F2 described giving an agent a prompt, doing something else, and returning only to inspect what it had built. F5 similarly reported scrolling through YouTube while Codex worked in the background. Reported signs also extended beyond interaction logs: F3 associated reduced engagement with tiredness and slouching.
No single behavior consistently represented effective or ineffective oversight. For example, short prompts could reflect either shallow engagement or an efficiently specified task, while extensive back-and-forth interaction could still consist of automatic correction requests. The diversity of reported signs therefore motivated examining oversight through multiple signals rather than presupposing a single behavioral marker.
3.3.Implications for the Controlled Study
The formative study was conducted with a limited pool of participants, and as such, a further controlled study was necessary to better examine how effective oversight might change during and across agentic interactions. Nevertheless, the formative study helped guide the design of our controlled study in four ways. First, participants most readily described reduced evaluation in AI-assisted programming. Coding agents could autonomously produce complete, executable outputs, while the correctness of those outputs could be objectively evaluated in the case of testable code generation. We therefore selected programming as a controlled setting in which behavioral oversight and task outcomes could be examined together.
Second, participants often relied on visible functionality to decide whether further evaluation was needed. We therefore designed two failure types that differed in whether the incorrect solution produced an external failure signal. Silent failures let us examine whether users initiated scrutiny independently, whereas visible failures let us examine how oversight changed when the need for human judgment was made explicit.
Third, participants described failures as events that could snap them back into active evaluation. We therefore studied repeated interactions to determine whether re-engagement remained limited to the visible failure or carried into subsequent tasks.
Finally, the reported signs of reduced oversight spanned interaction behavior, visual attention, bodily state, and subjective experience. We therefore captured multiple complementary measures in our main study: task outcomes to assess oversight effectiveness, interaction telemetry and gaze to capture observable behavior and attention, physiological signals to reflect bodily responses, and repeated self-reports to assess participants’ perceived engagement.
4.AI-assisted Programming Study
Guided by our formative findings, we designed a controlled laboratory study in which, we examined how users’ oversight changes across repeated interactions with a coding agent and whether a visible failure prompts them to re-engage more closely. We combined behavioral, gaze, physiological, and self-report measures to capture different aspects of their engagement. Smart glasses (Meta Aria Gen 2 glasses1
1
https://www.projectaria.com/
) were used to record gaze and egocentric activity during these interactions, while a Garmin watch was used to record physiological data. Figure 1 explains the basic study setup.
4.1.Participants
We recruited 44 participants who met two inclusion criteria: (1) self-reported basic proficiency in Python and (2) the ability to work comfortably on a laptop while wearing the Aria glasses without prescription glasses. Six were run as an initial pilot block and excluded from analysis as we changed the UX/UI interactions and task setup based on their feedback; two additional participants were excluded due to incomplete sessions (application/recording failures partway through the pipeline), leaving a final analytic sample of 36 participants. Participants ranged in age from 19 to 35 (
𝑀
=
25.4
) and reported their gender as male (23), female (12), or non-binary (1). Self-rated Python proficiency was predominantly intermediate (23 participants comfortable with functions, data structures, and common libraries), with 6 beginners (able to read and write simple scripts) and 7 advanced programmers (able to write complex programs comfortably). The study was approved by the Departmental Ethics Review Committee at [XXX University]. All participants provided informed consent and were compensated for their time.
4.2.Programming Interface
Figure 2.The programming interface used in the main study. (a): Participants could open the full task brief to review the task overview, requirements, and example input and output. Selecting “Got it—let’s build it!” returned them to the main workspace. (b): The main workspace provided (A) access to the task brief, (B) the provided pre-merge checks, (C) the code editor, (D) a panel for manually executing the code with custom inputs, (E) the coding agent’s responses, and (F) an input field for prompting the agent and asking follow-up questions.
We developed a desktop study interface that combined a task brief, a Python code editor, and an AI coding agent (Refer to Figure 2). The task brief presented all requirements for the current feature and could be reopened at any time. The editor contained scaffolded code with syntax highlighting and supported normal indentation and direct editing. Participants could also execute their own examples in a separate Run code area. The coding agent operated autonomously: when participants requested a change, the agent’s edit was applied directly to the file. Participants could subsequently inspect, modify, test, or discuss the resulting code with the agent.
Each task included hidden pre-merge checks. Participants were required to use the coding agent and obtain a passing result before handing off the feature. However, the introductory instructions explained that the checks covered only basic functionality and did not guarantee that every requirement or edge case was correct. This distinction allowed us to examine whether participants treated a passing check as sufficient evidence or performed additional verification through code inspection, customized examples, or follow-up questions.
4.3.Procedure
Table 1.Experimental conditions. Outcomes describe the initial agent-generated implementation before participant intervention.
Condition Code Checks
Defect visibility
No Defect (ND) Correct Pass
No defect present.
Silent Defect (SD) Incorrect Pass
Requires code inspection, reasoning, or targeted tests.
Blatant Defect (BD) Incorrect Fail
Revealed by a comment accompanying failed checks.
4.3.1.Conditions
Participants used the agent to implement a specified function within scaffolded Python code. We manipulated whether the agent-generated implementation contained a defect and whether the visible pre-merge checks exposed it. Table 1 summarizes the three conditions. Both the Silent Defect (SD) and Blatant Defect (BD) implementations were incorrect; however, only the BD implementation caused the provided pre-merge checks to fail.
In the No Defect (ND) condition, the agent-generated implementation correctly satisfied the task requirements and passed the visible pre-merge checks. In the Silent Defect (SD) condition, the implementation violated a less apparent requirement or failed on an edge case while still passing the visible checks. Participants could therefore detect the defect only by carefully inspecting the code, reasoning about the requirements, or conducting additional targeted tests. In the Blatant Defect (BD) condition, the implementation contained an error that caused the visible pre-merge checks to fail. A comment accompanying the failed checks explicitly exposed the issue, providing participants with an external signal that the implementation required further attention.
4.3.2.Tasks
Participants first completed two short practice problems to learn the interface. The first introduced the coding agent, editor, manual execution area, and pre-merge checks through a simple arithmetic function. The second provided Python scaffolding for an easy-to-solve list-filtering problem and allowed participants to practice editing code directly without the help of an agent. This task allowed us to confirm that all participants had the basic Python knowledge needed to complete the study.
The main study consisted of six Python programming tasks, organized into two sets of three. In each task, participants used the coding agent to implement a single function. The task descriptions and corresponding silent defects are provided in Appendix B. Participants could inspect and edit the generated code, execute it with their own inputs, ask the agent follow-up questions, and run the provided pre-merge checks. Participants proceeded to the next task after the pre-merge checks passed and they were satisfied with their solution.
Participants completed one task set in each of two rounds. Task-set order and the order of the no-defect, silent-defect, and blatant-defect conditions were counterbalanced across participants. The same condition order was repeated in the second round using the other task set. This design allowed us to compare oversight across defect conditions while reducing systematic effects associated with individual tasks and task-set order. After the first round, participants were shown the results for their first three solutions before continuing to the second round.
Figure 3.Main-study procedure. After demographics, smart glass fit and eye-tracking calibration, and two practice tasks, participants completed six programming tasks across two rounds. Each round contained one no-defect (ND), silent-defect (SD), and blatant-defect (BD) condition in randomized order. Participants received performance feedback after each round and completed a post-study interview.
4.4.Data Collection
We recorded participants’ prompts, agent responses, code changes, manual program executions, use of the pre-merge checks, access to the task brief, and task completion times. Participants wore Aria Gen 2 smart glasses to capture first-person video, gaze, and photoplethysmography (PPG). They also wore a Garmin smartwatch, which provided additional physiological measures. We separately recorded participants’ screens, keystrokes, and cursor movements. All data streams were synchronized with the study interaction logs. After each task, participants completed the NASA-TLX workload questionnaire. They also rated how thoroughly they had evaluated the agent’s suggestions and how confident they were that their final solution met all specified requirements.
At the end of the study, we conducted a brief semi-structured interview where participants reflected on how they interacted with the coding agent, evaluated its outputs, and results presented between rounds. We asked whether these results affected their confidence or verification behavior, whether they changed their approach during the second round, and what consequences might arise from relying on the agent without sufficiently examining its work.
5.Results
Table 2.Definitions and operationalizations of the three dimensions of oversight used throughout this paper.
Dimension
Working definition
How we measure
Perceived oversight
Participants’ assessment of how carefully and successfully they evaluated the agent’s work.
Self-reported confidence (1-5) and perceived thoroughness (1-5).
Behavioral oversight
Observable actions through which participants sought or evaluated evidence beyond the agent’s initial output.
Code inspection, requirement revisiting, manual execution, additional tests, questions to the agent, and evaluation–revision loops.
Effective oversight
Whether participants’ evaluation successfully identified and addressed a seeded defect.
Binary outcome: resolved vs. unresolved.
Based on how oversight appears in our data and how it can be measured, we developed three working definitions: perceived, behavioral, and effective oversight (Table 2). These capture, respectively, how users assess their own oversight, how oversight is shown in their behavior, and whether they actually resolved the silent defect. We use these definitions throughout the analysis and discussion.
5.1.General Findings
5.1.1.Sample and data quality.
The core cohort comprised 36 participants, each assigned six main tasks across two rounds: two Silent Defect (SD), two No Defect (ND), and two Blatant Defect (BD) tasks. This produced 216 task instances. Two participants lacked usable outcome data for their final task. One participant recorded no interaction with their final SD task. The other engaged substantially with their final BD task (10 agent turns and six pre-merge check attempts), but its outcome was not recorded. Both losses occurred at the sixth and final task position, suggesting end-of-session attrition rather than systematic loss under a particular condition. We retained these instances where their available data supported the analysis but excluded them from analyses requiring a final task outcome. This yielded 71 SD, 72 ND, and 71 BD tasks with usable outcomes (214 in total).
5.1.2.Task Effects
Silent-defect resolution increased from 10/36 tasks in Round 1 (27.8%) to 18/35 in Round 2 (51.4%). Because task sets were randomized across rounds, this difference was not attributable to a fixed task-set assignment. However, resolution still varied substantially by individual task, ranging from 8% for Conflicts to 75% for Billing. We therefore account for task identity in subsequent analyses. Full task-level results are reported in Appendix Table 5.
5.1.3.Individual differences.
Self-reported Python proficiency was associated with effective oversight. Silent-defect resolution increased from 16.7% among beginners (
𝑛
=
6
) to 39.1% among intermediate programmers (
𝑛
=
23
) and 57.1% among advanced programmers (
𝑛
=
7
; Spearman’s
𝜌
=
.34
,
𝑝
=
.041
). In contrast, self-reported experience with agentic AI tools was not associated with defect resolution (
𝑝
>
.48
). Participants who frequently delegated multi-step tasks to AI were therefore no more thorough or effective than less frequent users. We also observed no association between gender and either measure (all
𝑝
>
.56
; 23 men, 12 women, and one non-binary participant).
5.2.Confidence and Perceived Thoroughness Capture Different Aspects of Oversight
After each main task, we asked participants to rate, on a five-point scale, (1) how confident they were that their final solution met all the requirements and (2) how thoroughly they had checked the coding agent’s suggestions. Although both measures concern participants’ perceptions of their own oversight, they showed notably different relationships with behavior and outcome.
5.2.1.Confidence remained high when defects were unresolved
Self-reported confidence did not reliably distinguish whether participants had detected a silent defect. Among the 43 failed silent-defect tasks with valid confidence ratings, 34 (79%) received a confidence rating of 4 or 5 out of 5. Participants were therefore not simply uncertain when they handed off an incorrect solution; they were frequently highly confident that it met the requirements.
Figure 4.Confidence and task duration for resolved and unresolved silent-defect instances. Self-reported confidence remained similarly high across outcomes and rounds (left), whereas resolved defects involved substantially more time on task than unresolved defects (10.8 vs. 5.8 minutes,
𝑝
<
.001
; right).
Confidence remained high in both rounds. Mean confidence on silent-defect tasks decreased only slightly, from 4.17 in Round 1 to 4.09 in Round 2, and this difference was not statistically significant. Even after participants saw the broader release results for their Round 1 implementations, we found no evidence that their confidence became better calibrated in Round 2. As depicted in Figure 4 confidence remained high for both passed and failed silent defects: 4.30 versus 4.12 in Round 1 and 4.22 versus 3.94 in Round 2, respectively. Task behavior provided a clearer distinction. Participants spent nearly twice as long on silent-defect tasks in which they recognized the defect (M = 10.8 min) than on those in which they missed it (M = 5.8 min; Mann-Whitney U, p < .001). Thus, confidence provided little indication of an impending oversight failure, even when participants’ task behavior differed.
5.2.2.Perceived thoroughness tracked within-person changes in verification
Perceived thoroughness showed a different pattern. Across all 214 main-task observations, a participant tended to report greater thoroughness on tasks where they spent more time than was typical for them (
𝑟
within
=
0.258
,
𝑝
<
.001
). They also reported greater thoroughness when they performed more manual code executions than was typical for them (
𝑟
within
=
0.158
,
𝑝
=
.021
). In contrast, the corresponding between-person relationships were not significant. A participant who generally used higher thoroughness ratings was not necessarily more behaviorally thorough than another participant who generally used lower ratings.
This within-person relationship also extended to silent-defect outcomes. Caught silent-defect tasks received a mean thoroughness rating of 4.14, compared with 3.49 for missed tasks (
𝑟
=
0.270
,
𝑝
=
.023
). A stricter within-participant comparison produced the same pattern. Among the 16 participants who caught one silent defect and missed the other, participants rated themselves an average of 0.93 points more thorough on the task in which they caught the defect (
𝑝
=
.014
); 10 of the 16 participants gave a higher thoroughness rating specifically to their caught task, 4 rated themselves equally thorough on both, and 1 rated themselves less thorough on the caught task. The changes relative to a participant’s own typical rating carry meaningful information: when participants perceived themselves as being more thorough than usual, they generally spent longer, conducted more manual checks, and were more likely to detect the silent defect.
5.3.Behavior Following an External Defect Warning
The ND, SD, and BD conditions were designed to examine how participants behaved before and after receiving an external signal that the agent could be wrong. We expected participants to approach an SD task similarly to an ND task before encountering such a signal. We further expected the failed pre-merge check and accompanying comment in the BD condition to increase scrutiny of subsequent agent output, with the strongest effect occurring on the task immediately following BD.
At an aggregate level, encountering a BD warning did not improve subsequent silent-defect detection. Participants caught 38.89% of silent defects when SD occurred before BD and 40% when SD occurred after BD. These catch rates were statistically indistinguishable (
𝜒
2
=
0.00
,
𝑝
=
1.00
). Task duration also did not differ significantly depending on whether SD occurred before or after BD (
𝑝
=
.170
). However, because different tasks appeared in different sequence positions and varied substantially in difficulty, the aggregate before-versus-after comparison cannot cleanly separate the effect of prior BD exposure from the effect of task identity.
However, the overall before-and-after comparison masked differences in the timing of the subsequent silent-defect task. When a silent defect immediately followed a visible failure (BD
→
SD), participants detected 50% of defects (12/24 tasks; 12 participants). When one correct task intervened (BD
→
ND
→
SD), detection fell to 9.1% (1/11 tasks; six participants). The difference was supported by an exact participant-level permutation test (p=.046). A task-level Fisher’s exact test produced a similar result (p=.023); we treat this only as a sensitivity analysis because multiple tasks were contributed by the same participants. Because task identity differed across sequence positions, we additionally compared performance after residualizing each task by its own mean resolution rate. The adjacent-versus-gap difference remained significant after this adjustment (
𝑡
=
2.44
,
𝑝
=
.020
). This comparison is supported by within-task overlap, as Equipment and Waitlist appeared in both sequence groups.
5.4.Input inactivity reflected effective oversight
Across 21 keyboard and cursor measures, silent-defect and no-defect tasks differed in completion time but not in how much participants typed or moved after accounting for task duration. However, silent defects that were resolved contained a substantially greater proportion of time without typing than those that were missed (.622 vs. .425,
𝑝
<
.001
, Cohen’s
𝑑
=
1.29
). Longer gaps without keystrokes and greater variability in typing rhythm followed the same descriptive pattern, whereas keystroke rate, deletions, navigation inputs, cursor travel, and cursor speed did not distinguish the outcomes. Although periods without input are compatible with participants pausing to read the implementation, revisit the requirements, or reason about possible edge cases. Evaluation effort may therefore be more visible when interaction stops than in the amount of observable activity.
5.4.1.Gaze dynamics did not provide a reliable task-level marker.
We analysed 71 silent-defect tasks from 36 participants, comprising 28 resolved and 43 unresolved defects. In pooled comparisons, resolved tasks had fewer fixations per minute (146.97 vs. 167.36,
𝑝
=
.042
), fewer saccades per minute (169.04 vs. 196.66,
𝑝
=
.034
), and smaller mean saccade amplitudes (
7.14
∘
vs.
8.42
∘
,
𝑝
=
.009
). However, these patterns did not consistently persist within participants. Among the 16 participants who resolved one silent defect but missed another, saccade rate and amplitude no longer differed between outcomes (all
𝑝
≥
.064
), while fixation rate reversed direction: participants produced 7.39 more fixations per minute during resolved tasks (
𝑝
=
.008
).
After accounting for task duration, round, task position, and repeated observations, the estimated within-participant association between fixation rate and defect resolution remained positive but uncertain (OR
=
1.67
, 95% CrI
[
0.87
,
3.22
]
). In contrast, task duration showed a stronger association with resolution (OR
=
5.32
, 95% CrI
[
2.50
,
11.35
]
). General gaze dynamics may therefore contain some information about changes in an individual’s behavior, but they did not provide a reliable standalone marker of effective oversight.
5.4.2.Physiological signals did not distinguish conditions or outcomes.
The PPG signal captured by the Aria Gen 2 glasses was too noisy for reliable task-level analysis and was therefore excluded. We instead analysed cardiac and respiratory measures recorded by the Garmin watch wearable. None reliably differed across the ND, SD, and BD conditions or between resolved and unresolved silent defects. However, physiological arousal declined gradually over the session, suggesting that these measures were more sensitive to participants settling into the study environment than to the evaluation demands of individual tasks.
Full feature definitions, preprocessing procedures, statistical models, and results are provided in the supplementary materials.
5.5.Understanding Behavioral Oversight in Agentic Interactions
Our analysis was informed by prior work on interaction-strategy mining (Fern et al., 2007) and taxonomies of AI-assisted programming (Mozannar et al., 2024). We manually coded 216 participant-authored messages to the agent, as well as code entered in the manual execution panel, to identify participants’ interaction and verification strategies. Message codes captured requests for implementation, verification, explanation, diagnosis, and correction, while scratch code indicated attempts to test the implementation with participant-generated inputs. The first author developed the codebook using a diverse pilot sample and then coded the complete dataset. A second author independently coded a stratified 24% sample, achieving an agreement of 75% (linear-weighted Cohen’s
𝜅
= 0.632). Disagreements were discussed and resolved. When the meaning of a message or action was unclear, we examined the corresponding first-person video segment for context. We then combined these manually coded data with automatically recorded actions, including task-brief revisits, manual program executions, pre-merge checks, code edits, and task completion. This produced a chronological record of oversight behavior for each participant–task pair.
Figure 5.Progression of the 71 silent-defect task instances through the oversight pipeline. The main pathway shows task-relevant evaluation, production of defect-revealing evidence, correct interpretation, and resolution. Purple branches indicate where task instances stopped progressing. Two additional defects were resolved through non-targeted evaluation: one through generic verification and one through direct code inspection.
5.5.1.Where Oversight Breaks Down
To characterize how participants evaluated the agent’s output before completing a silent-defect task, we traced all 71 task instances through these stages (Figure 5). To illustrate, consider the Billing task, in which participants asked the agent to reconcile a member’s charges into a running total, distinguishing between approved, refunded, and cancelled amounts. The seeded defect ignored refunds instead of subtracting them. Generic evaluation, such as rerunning the provided example or asking the agent to check its code, would not necessarily expose this defect. Even a task-relevant test involving cancelled charges would not reveal it. In contrast, testing an approved charge together with an equal refund would expose the error because the balance should be zero. Effective oversight required participants to notice this mismatch, correct the refund logic, and verify the revised solution. The defect could also be identified directly by inspecting the code.
Most instances (65/71, 91.5%) included at least one evaluative action beyond the mandatory pre-merge check, whereas six relied on that check alone. The relevance of these additional actions varied substantially. In 23 instances, evaluation remained generic, such as rerunning an existing example or asking the agent to "check" the solution without specifying what to examine. In 40 instances, participants evaluated risks associated with the task requirement
…(truncated)