Nowadays in modern embodied AI, when we train robots through imitation learning or behavioral cloning, we often rely on a human controlling a robot by using camera headsets, data gloves, or haptic rigs. The implicit assumption in many machine learning pipelines is that the human acts as a perfect, smooth "oracle." But in reality, as we saw in previous journals (A Taxonomy of Teleoperator Noise (2026)), the human body is a biological system subject to physical fatigue. Microtremors, cognitive hesitation, little pauses, and ergonomic discomfort can affect the correct trajectory we want to obtain from the data. We call these unwanted variables "teleoperator noise" (recommended: The Human Factor in Teleoperation: A Taxonomy of Teleoperator Noise (2026) ).
When training is purely offline, these human errors keep accumulating, leading to compounding errors at the point where the robot doesn't know how to fix itself if it makes a mistake during the task.
In the machine learning literature, there is one algorithm that gets brought up as the definitive mathematical cure to this problem ("the noise"): DAgger (Dataset Aggregation). On paper, DAgger solves distribution shifts by putting the human back into the loop while the robot is doing the task. At this point it looks great, but then a paradox emerges: What happens when the very mechanism meant to fix policy drift relies on an operator who is tired, lagging, or jittering?
This journal takes a closer look at DAgger. It's not just an algorithmic explainer, but a case study of what actually happens when you put a noisy human inside the training loop.
DAgger in one paragraph
Introduced by Ross, Gordon, and Bagnell, Dataset Aggregation (DAgger) was designed to tackle the fundamental flaw of Behavioral Cloning: in standard BC, the model only learns from clean trajectories recorded offline, so any small execution error places the robot in a state it was never trained on, where its performance degrades quickly. [1]
Rather than relying on a fixed batch of human demonstrations and hoping for the best, the robot is allowed to execute its current policy live. While it runs, a human expert observes and provides the correct action for whatever state the robot is actually in, not only the states contained in the original demonstrations. Those new corrections get folded into the training set, the policy gets retrained, and the whole system starts running again. After doing this enough times, the dataset starts looking less like "what the human did" and more like "what the robot actually sees when it's running on its own during the activity," which is exactly the gap that imitation learning never closes.
The assumption DAgger quietly makes
Here is the catch, though: all of DAgger's mathematical elegance and theoretical guarantees rest on one assumption nobody writes down explicitly, and we already introduced the idea that the human supervisor is a perfectly state-aware oracle, someone who just returns the correct label on demand every single time, no questions asked. We already know that's not how it works. It's still the same operator using a biological system we've been talking about this whole series. Putting a human in "the loop" doesn't make them stop being human, no matter how expert they are at an activity. [2]
And here is the paradox: that "noise" doesn't just disappear once the human is back in the loop it moves. Instead of living inside a recorded demonstration, sitting there under calm, baseline conditions, all that jitter and lag now get injected straight into the live correction signal. It is worth noting that this is not a minor detail: when a noisy or delayed correction occurs, it tends to land precisely in the states where the robot is already struggling. That is the point in the trajectory where the supervisor matters most, and it is also the point where the operator is asked to respond fastest and under the greatest pressure.
This concern is raised directly by the authors of HG-DAgger, who note that asking an operator to intervene mid-task, while the robot is still executing and without full control of the system, degrades label quality due to the lag between the operator's input and the robot's response [2]. So no, DAgger doesn't get rid of the noise, it relocates it straight into the exact point when the policy needs it the most.
Query frequency as a fatigue multiplier
Consider how vanilla DAgger (the original algorithm) operates in practice. It does not request occasional corrections; it queries the supervisor at essentially every state the robot visits. This means the operator is repeatedly switching in and out of control while remaining alert for the entire session. The result is sustained micromuscular and cognitive load, the same load described in the Taxonomy Journal, now applied continuously for the full duration of the recording.
This is not incidental. As these systems were deployed at scale across robot fleets, operator fatigue emerged as one of the main reasons teams moved away from constant supervision. [3][5] Requiring a human to monitor a policy without interruption leads to operator burnout, and a fatigued operator produces lower-quality corrections, which undermines the purpose of the intervention in the first place.
Human-gated variants
One fix that came out of this is pretty intuitive: let the human decide when to step in instead of the algorithm forcing a correction at some arbitrary moment. That's basically what HG-DAgger does. Rather than having the robot continuously request labels, the supervisor observes and intervenes only when they judge it necessary. [2]The distinction matters: it prevents corrections from being triggered before the operator has fully assessed what the robot is doing wrong, which is precisely the condition under which cognitive hesitation enters the correction signal.
Budget-aware and risk-gated variants
A separate family of methods approaches the problem from the opposite direction. Instead of relying on the human to decide when to intervene, the robot estimates how risky or unfamiliar its current state is and requests supervision only when that estimate crosses a defined threshold. That's the idea behind things like SafeDAgger[5], EnsembleDAgger[6], LazyDAgger[7], and ThriftyDAgger[4]. ThriftyDAgger in particular literally budgets the interventions; it only asks for help at states that are genuinely novel or genuinely risky, which cuts down the total supervision load a lot compared to plain DAgger. [4] And when you scale this up to fleets, where a handful of humans are supervising a bunch of robots at once, human attention becomes the actual bottleneck, since human attention is limited and fatigue-prone in a way robots simply aren't. [3]
Looking at all of this through the lens of the original taxonomy, these aren't just clever tricks to save on training time. They're noise-management tools. By controlling when and how often a human is asked to react, they cap how much cumulative fatigue and cognitive load can build up in a single session, which is exactly the same logic behind putting hard limits on recording blocks back in the operational levers journal.
Nurvai’s closing thoughts
At the end of the day, DAgger isn't really a story about an algorithm; it's a story about data. Every version of it, from the original paper to HG-DAgger to the budget-aware variants, is answering the same underlying question: what does the dataset need to look like for a robot to actually learn the task, not just imitate a handful of clean demonstrations? That's the part that matters more than the math.
Whether the correction comes constantly, only when the human decides it's necessary, or only when the robot itself flags a risky state, what's really being decided every time is which states get recorded, under what conditions, and how much human noise rides along with them.
That's the lens we try to bring to every dataset we help collect. It's not enough to know that a model needs “more data” or “on-policy corrections”; you need to understand what's actually happening to the human providing that data tired or fresh, rushed or steady because that's what ends up baked into the states and actions the model trains on.
Understanding operator noise, and understanding how different collection strategies move that noise around instead of eliminating it, is exactly what lets us help teams collect datasets that actually hold up once the robot is running on its own.
If you are interested in learning more about how human noise shapes teleoperation data, in exploring these dynamics in greater depth, or in sourcing robotics datasets collected and curated by our team, feel free to connect with us: Free Consultation with Nurvai
Connect with us on socials LinkedIn | X
Picture from news.mit.edu
References
[1] Ross, S., Gordon, G. J., & Bagnell, D. (2011). A reduction of imitation learning and structured prediction to no-regret online learning. Proceedings of the 14th International Conference on Artificial Intelligence and Statistics (AISTATS). A reduction of imitation learning and structured prediction to no-regret online learning
[2] Kelly, M., Sidrane, C., Driggs-Campbell, K., & Kochenderfer, M. J. (2019). HG-DAgger: Interactive Imitation Learning with Human Experts.Interactive Imitation Learning with Human Experts
[3] Hoque, R., Chen, L. Y., Sharma, S., Dharmarajan, K., Thananjeyan, B., Abbeel, P., & Goldberg, K. (2022). Fleet-DAgger: Interactive Robot Fleet Learning with Scalable Human Supervision. Conference on Robot Learning (CoRL). Fleet-DAgger: Interactive Robot Fleet Learning with Scalable Human Supervision
[4] Hoque, R., Balakrishna, A., et al. (2021). ThriftyDAgger: Budget-Aware Novelty and Risk Gating for Interactive Imitation Learning. ThriftyDAgger: Budget-Aware Novelty and Risk Gating for Interactive Imitation Learning
[5] Zhang, J., & Cho, K. (2016). Query-Efficient Imitation Learning for End-to-End Autonomous Driving. arXiv:1605.06450 Query-Efficient Imitation Learning for End-to-End Autonomous Driving
[6] Menda, K., Driggs-Campbell, K., & Kochenderfer, M. J. (2019). EnsembleDAgger: A Bayesian Approach to Safe Imitation Learning. arXiv:1807.08364 EnsembleDAgger: A Bayesian Approach to Safe Imitation Learning
[7] Hoque, R., Balakrishna, A., Putterman, C., Luo, M., Brown, D. S., Seita, D., Thananjeyan, B., Novoseller, E., & Goldberg, K. (2021). LazyDAgger: Reducing Context Switching in Interactive Imitation Learning. IEEE CASE 2021. arXiv:2104.00053 LazyDAgger: Reducing Context Switching in Interactive Imitation Learning
