Self-improving agent harnesses: Recursive Language Models, Continual Harness and Prime Agent

Instead of stuffing everything into the model's context, let the model hold its context in variables, write code over it, call copies of itself, and edit its own notes, skills and helpers as it learns.

Source: Yacine Mahdid, X post (23 Oct 2026), announcing a 2h36m walkthrough with Seth Karten (Prime Intellect): “Prime Agent: Self-Improving RLM Harness”. I couldn't watch the video, so this module is built from the primary written sources: the Prime Agent paper, repo and launch post, the RLM paper and posts, and the Continual Harness paper. All are saved in sources/10-*.

Going deeper: everything below is the full version. On a first read, look at the three-ideas table, the demo and the robotics section.

1. Summary

2. Why this matters for robotics

• Continual Harness was born embodied. It came from the Gemini Plays Pokémon experiments, where human-in-the-loop harness refinement helped the agent become, per the paper, the first AI system to complete Pokémon Blue, Yellow Legacy on hard mode, and Crystal without a lost battle. Continual Harness then removed the human from that loop. Long horizons plus partial observability is the robot setting.
• Reset-free learning is the real-world constraint. Prompt-optimisation methods need episode resets. Continual Harness adapts within a single run. A home robot can't reset the kitchen between attempts.
• Skills libraries and sub-agents mirror robot stacks. Many robot systems already use an LLM planner that calls skills (grasp, navigate, open-door) and delegates to specialised policies. A harness that can add and refine its own skill descriptions and helper specs is a software version of that stack learning on the job.
• Programmatic context fits sensor data. An hour of robot logs (joint states, detections, events) is far too big for a context window. Holding it as a variable and querying it with code ("when did the gripper last slip?") is the RLM pattern.
• Bounded autonomy is a safety envelope. Prime Agent's autonomous mode runs within turn, token and time budgets with quality gates. Robots need the same idea with physical limits. See Module 03, levels of automation.

3. Deep dive: Recursive Language Models

The problem. Per-token cost rises with context length, and quality tends to fall as contexts grow. This is called context rot. Common fixes: summarise or compact the history (lossy), or pass state between agents through files.

The RLM move. Don't give the model the context at all. Give it the query, plus a REPL in which the context is pre-loaded as a variable. The model (the "root" LM, depth 0) writes code to peek at, search, slice and transform that variable. It can also call sub-LMs, fresh copies of itself, on pieces of it, and combine their answers. From the outside it's a drop-in replacement for a normal completion call.

# the shape of an RLM call (illustrative, not the library's exact API)
answer = rlm.completion(query, context=huge_text)   # same interface as an LM call

# inside, the root model might write:
chunks  = split(context, 100_000)
notes   = llm_batch([f"Find mentions of X in:\n{c}" for c in chunks])   # parallel sub-LMs
answer  = llm(f"Combine these findings to answer: {query}\n{notes}")

• No summarisation, so no forced loss. The full context stays available in memory. The model chooses what to read.
• Test-time compute becomes programmable: loops, filters, parallel fan-out.
• Prime Intellect's variant (in their verifiers library): only sub-LLMs get tools beyond the REPL, so tool output tokens don't flood the root. Answers are returned via a variable. An llm_batch function runs sub-calls in parallel.

Reported results (RLM paper abstract):

• Handles inputs up to two orders of magnitude beyond the model's context window.
• With GPT-5, it beats common scaffolds at comparable cost. The median improvements across four long-context tasks were 26% over compaction, 130% over CodeAct with sub-calls, and 13% over Claude Code.
• A small post-trained model, RLM-Qwen3-8B, beat its base Qwen3-8B by 28.3% on average.

4. Deep dive: Continual Harness, an agent that edits its own harness

The observation. In the Gemini Plays Pokémon runs, people kept improving the harness. In the hardest stages, the agent itself began iterating on its strategy through long-context memory. Continual Harness formalises and automates that loop.

The formalism. The harness state is

H = (ρ, G, K, M)   # prompt, sub-agents, skills, memory

Starting from only a minimal environment interface, the agent alternates between acting and refining H, drawing on any past trajectory data, all without resetting the environment.

• Result (paper abstract): on Pokémon Red and Emerald across frontier models, starting from scratch, it substantially cut button-press cost compared with a minimal baseline. It also recovered most of the gap to a hand-engineered expert harness. Gains depended on model capability.
• Closing the loop with the model: an online process-reward co-learning loop. An open-source agent's rollouts through the refining harness are relabelled by a frontier teacher and used to update the model, again without environment resets. It's the distillation pattern of Module 04, run continuously.

In Prime Agent, the harness state lives in the REPL as rlm.harness, with one create/read/update/delete surface for all four kinds (prompt notes, memories, skills, sub-agent specs), persisted to disk.

/refine reads the trajectory and applies the smallest relevant edit, recording its trigger and outcome. The planning runs in the background, and the apply step blocks only briefly at a turn boundary. The base system prompt is immutable, and refinements can be rolled back by ID.

Why "small, evidence-backed, reversible" matters: self-modifying systems can drift, the same failure as reward hacking in Module 07. Small diffs, recorded evidence, a fixed core and rollback are the harness-level equivalent of a KL leash.

5. Deep dive: Prime Agent, putting it together

Compare the two ways of beating long context in this course. Module 05 changes the architecture (Kimi K3's fixed-size linear-attention memory). RLMs change the scaffolding, so the model manages context with code. They're complementary. External, addressable memory also echoes the Neural Turing Machine (Module 06, Day 17), now with Python as the memory controller.

6. Deep dive: results, and how to read them

• ARC-AGI-3: the paper reports raising RHAE Best@1 from 30% to 95.5%. The launch post attributes the best result to Opus 5 inside Prime Agent, notes the ARC-reported human-expert baseline of 95.4%, and gives three runs of 95.0, 95.2 and 95.5, with Best@3 99.97% and all 183 levels completed. The only ARC-specific change was the task prompt.
• Token efficiency: the post reports higher maximum scores than the models' native harnesses at lower token use. Its explanation is that running functions over data costs fewer tokens than reading the data through tools.
• Long-context suite: using the open-weights GLM-5.2 in Prime Agent against closed models in their own harnesses, the post shows Prime Agent competitive or ahead on several tasks. One example: OOLONG (128k), 0.700 for Prime Agent with GLM-5.2 vs 0.420 for Pi-mono with sub-agents.
• Also reported: results on GPU-kernel generation, emulator construction, autonomous nanoGPT speedruns and Factorio. In Factorio, refinement enabled continuous technology progression and dedicated sub-agents parallelised work.

7. Deep dive: safety and bounds

• Not a sandbox: the README warns that Prime Agent executes model-generated Python and project commands with your user permissions. Its process isolation is for recovery, not security. Use disposable clones or worktrees, trusted repos and skills only, and an external sandbox for untrusted code.
• Bounded autonomy: autonomous mode stops at configured turn, token and time limits. The README notes that passing a gate checks only what that gate verifies, and that hitting a limit doesn't mean success.
• Self-editing risk: refinement edits only the supplemental layer, keeps the base prompt immutable, records history and supports rollback. Review refinements as you would code changes.
• Communication scope: agent-to-agent messaging is restricted to parent, sibling and child sessions.

8. Playbook

1. Try the minimal RLM first: rlm-minimal. Put a long document in a REPL variable and let a model answer questions by writing code over it.
2. Compare against "paste it all in" and "summarise then answer" on the same questions. Measure accuracy and total tokens.
3. Run Prime Agent in a disposable worktree. Give it a goal, a quality gate (tests) and tight turn, token and time budgets.
4. Watch /refine: read each proposed edit, keep the good ones, and roll back the rest. Treat harness state like code under review.
5. Robotics exercise: log a simulated robot run (states, events, failures) to a file, load it as a variable, and have an RLM-style agent answer "why did grasp attempt 7 fail?" by querying the log with code.

9. Self-check

10. Glossary & further reading

• Karten, Zhang et al., Prime Agent: A Self-Improving RLM Harness (arXiv 2608.23552) · repo · launch post · project page
• Zhang, Kraska & Khattab, Recursive Language Models (arXiv 2512.24601) · original blog post · library · minimal implementation
• Karten et al., Continual Harness: Online Adaptation for Self-Improving Foundation Agents (arXiv 2605.09998)
• Prime Intellect, Recursive Language Models: the paradigm of 2026 · verifiers · prime-rl
• The 2h36m video walkthrough (Yacine Mahdid with Seth Karten)
• In this course: 05 (architectural long context), 06 Day 17 (external memory), 07 (verifiers and process rewards), 04 (teacher relabelling), 03 (levels of autonomy), 08 Stage 10 (evals)