How much to help: human–AI and human–robot interaction
An AI tutor for kids raises the same question every assistive robot faces: how do you help without taking over?
- Summary
- Why this matters for robotics
- Deep dive: levels of automation
- Deep dive: shared autonomy and policy blending
- Deep dive: inferring intent, and when the human changes their mind
- Deep dive: ask, act or wait
- Deep dive: don't automate your own learning
- Playbook
- Self-check
- Glossary & further reading
1. Summary
TL;DR
Linus Ekenstam argues that the biggest mistake in AI education is automating the part where the child learns: the questions, wrong turns and retries. Gakku's AI companion, Raku, coaches kids aged 6–16 in writing, speaking and thinking through live conversation. The hard design problem isn't generating good replies. It's following the learner's intent, coping when they interrupt or change their mind, keeping the conversation and screen in sync, and deciding how much to help.
- Core claim: the time spent thinking is the product's value. When AI can generate any answer, helping someone develop their own answer is what matters.
- Interaction design: the child can interrupt, revise an idea or make a choice on screen. The system must adapt and keep conversation, visuals and mission progress consistent.
- Agent decisions: follow intent, remember context, and choose whether to ask, act or wait.
- The calibration problem: too little help and the child gets stuck. Too much and the system makes the decisions the child needs to practise.
- About gakku: made by Algorithm of Learning Inc. Positioned as an AI writing tutor with persistent memory, a structured "missions" curriculum, and alignment to CCSS, CEFR and GCSE standards (per its site).
- Ekenstam is testing it with his six-year-old and plans to report back.
2. Why this matters for robotics
Track: LLM + Robotics. The source is about an LLM tutor. "How much should the AI help?" is the same question in human–robot interaction (HRI), where robotics has decades of formal work. This module uses that work to answer it for both.
| In the tutor | In a robot |
|---|---|
| Child is writing an essay | Person teleoperating a robotic arm, or using a wheelchair-mounted arm |
| Raku guesses what the child is trying to say | Robot infers which object the user is reaching for |
| Too much help: writes it for them | Robot grabs the wrong cup, and the user loses control |
| Too little help: child gets stuck | User fights a twitchy 7-joint arm with a 2-axis joystick |
| Child changes their mind mid-sentence | User switches target mid-reach |
| Ask, act or wait | Request clarification, take over, or keep observing |
| Conversation, screen and progress stay in sync | Robot's world model, the UI and the task state stay consistent |
Robotics gives these ideas maths you can implement. The rest of the module walks through them.
3. Deep dive: levels of automation
Sheridan and Verplank (1978) described automation as a scale, not a switch. A simplified version:
- Human does everything. The machine offers nothing.
- Machine suggests options. Human picks.
- Machine recommends one option. Human approves.
- Machine acts unless the human vetoes in time.
- Machine acts and informs the human afterwards.
- Machine acts alone and ignores the human.
Ekenstam's point maps onto this scale. A learning product should sit around levels 2–3 for the skill being learned, even if it could do level 6. A surgical robot sits low for cutting decisions but high for tremor filtering. The right level differs per sub-task, and good systems set it per sub-task rather than globally.
Known automation failure modes to design against:
- Complacency: humans stop monitoring a system that's usually right.
- Skill decay: operators lose the ability to take over, as pilots can with autopilot.
- Mode confusion: the human doesn't know which level the machine is at.
4. Deep dive: shared autonomy and policy blending
In shared autonomy, the human's input and the robot's own policy are combined every control cycle. The classic formulation is policy blending (Dragan & Srinivasa, 2013):
u = α · u_robot + (1 − α) · u_human α ∈ [0, 1]u_human: what the joystick says.u_robot: what the robot would do to reach the goal it thinks you want.α(arbitration): how much the robot takes over. Usually it isn't fixed. It rises with the robot's confidence about your goal.
Their key finding: "aggressive" assistance is great when the robot guesses right and frustrating when it guesses wrong. Users prefer a system that defers when uncertain. That's Ekenstam's "too much help" in engineering form. Confidence here means a calibrated probability, which links straight back to Module 01's calibration section.
Interactive: a robot assisting a reach
The simulated user (thin line) heads for goal B, with a noisy hand. The robot doesn't know the target. It infers it from the motion and blends in its own push toward its best guess. The thick line is the actual path.
5. Deep dive: inferring intent, and when the human changes their mind
The robot keeps a belief, a probability for each possible goal, and updates it with every movement. A standard model treats the human as noisily rational: people mostly move toward their goal, with noise. In Bayesian form:
P(goal | motion) ∝ P(motion | goal) · P(goal)Motions pointing toward goal g raise P(g). The demo uses a simple version: a softmax over how well recent motion aligns with each goal's direction.
Changing your mind is the hard case. If the belief only accumulates evidence, it gets "stuck" on the first goal and fights the user. Two fixes:
- Forgetting: decay old evidence so recent motion dominates. Toggle it in the demo above. Without it, a late change of mind leaves the robot locked on the old goal at 100% confidence.
- Explicit goal-switch models: give a small prior probability to "the user changed goal" at every step. This is a hidden Markov model over goals, closely related to the Markov chains in Module 02.
This is the robotic version of Ekenstam's design problem: "a learner can interrupt, revise an idea or make a choice… Raku has to adapt while keeping everything in sync." In both cases the system's belief about intent has to stay revisable.
6. Deep dive: ask, act or wait
Ekenstam names three choices. Decision theory gives a clean way to choose. Compare expected costs:
| Option | Cost | Best when |
|---|---|---|
| Act | P(wrong) × cost of a wrong action | Confident and actions are reversible |
| Ask | Interruption cost (annoyance, lost flow) | Uncertain and a wrong action is costly |
| Wait | Delay cost. More observation may resolve the uncertainty for free. | Uncertain but the person is still giving signal |
Horvitz's mixed-initiative principles (1999) say the same thing for software assistants. Act only when expected value beats asking, and asking beats doing nothing. A robot handing over a knife should ask or wait far more readily than one passing a pillow.
In a learning product there's a twist: a correct action can still be the wrong choice, because the user's struggle has value. The cost function has to include "the learner didn't get to practise this".
7. Deep dive: don't automate your own learning
Ekenstam's thesis applies to you, learning robotics with an AI at your side. Research on learning backs it:
- Desirable difficulties (Bjork): effortful retrieval, spacing and making errors produce more durable learning than smooth, easy study.
- Scaffolding (Wood, Bruner & Ross, 1976) and the zone of proximal development (Vygotsky): help should cover only what you can't yet do alone, and fade as you improve. That's α scaled by need, much like the demo.
- Generation effect: answers you produce yourself are remembered better than answers you read.
Practical consequence for this course: on TensorTonic, attempt before you peek. When you ask an AI for help, ask for a hint or the next question, not the solution. Use AI to generate drills, check your reasoning and explain your mistakes, not to skip the struggle.
8. Playbook
Designing an assistive robot or agent:
- Split the task into sub-tasks. Pick a level of automation for each, not one global setting.
- Keep an explicit, revisable belief over what the human wants, with forgetting or goal-switch modelling.
- Scale assistance by calibrated confidence. Defer when unsure.
- Decide ask / act / wait by expected cost, weighting irreversible actions heavily.
- Always give the human an override that wins immediately, and show what mode the system is in.
- Evaluate with real users: success, time, and perceived control. Fast-but-frustrating is a failure.
Learning with AI: hints over answers, attempt before you peek, explain it back in your own words, and fade the help over time.
9. Self-check
In u = α·u_robot + (1−α)·u_human, what goes wrong with α fixed at 0.9?
When the robot's goal guess is wrong, the human has only 10% authority and can't correct it. Users experience this as the robot fighting them. Scale α by confidence instead.
Why does an intent model need "forgetting" or a goal-switch probability?
Without it, early evidence dominates. If the user changes their mind, the belief stays locked on the old goal and the assistance pushes the wrong way.
A robot is 70% sure you want the hot coffee mug and 30% the empty glass. Ask, act or wait?
Probably wait (or ask, if the user has stopped moving). 30% wrong times "grabbed a hot drink unasked" is a high expected cost, and more motion will likely resolve it cheaply.
How does Module 01 (calibration) connect to shared autonomy?
Arbitration uses the robot's confidence. If that probability is badly calibrated, e.g. it says 90% when it's right 60% of the time, the robot takes over far too often.
Which level of automation should a learning tool use for the exact skill being learned?
Low: suggest or ask, levels 2–3. The learner has to make the decisions. Higher automation is fine for surrounding chores such as formatting or logistics.
10. Glossary & further reading
| Term | Meaning |
|---|---|
| HRI | Human–robot interaction: the field studying how people and robots work together |
| Shared autonomy / shared control | Human and robot inputs combined continuously to control one system |
| Arbitration (α) | How much weight the robot's command gets versus the human's |
| Intent inference | Estimating the human's goal from their actions |
| Noisily rational model | Assumes humans mostly act toward their goal, with random deviations |
| Mixed initiative | Either party (human or machine) can take the lead at different moments |
| Levels of automation | Scale from fully manual to fully autonomous (Sheridan & Verplank) |
| Scaffolding | Temporary support that fades as the learner gains skill |
| Desirable difficulty | Effort during learning that improves long-term retention |
- Dragan & Srinivasa, “A policy-blending formalism for shared control”, IJRR 2013
- Javdani, Srinivasa & Bagnell, “Shared autonomy via hindsight optimization”, RSS 2015
- Sheridan & Verplank, “Human and computer control of undersea teleoperators”, 1978 (levels of automation)
- Horvitz, “Principles of mixed-initiative user interfaces”, CHI 1999
- Bjork & Bjork, “Making things hard on yourself, but in a good way: creating desirable difficulties”, 2011
- gakku.ai · Original post and demo video
- Related: Module 01 (calibrated confidence), Module 02 (Markov chains, practice habits), Module 04 (fast voice commands and stopping distance), Module 05 (learned forgetting in KDA attention)