Self-Direction · continual learning for LLM agents

Don’t Forget What You Just Learned: Self-Directed Repair Turns Agent Mistakes Into Lasting Lessons

Yuval Kansal1,* Karen Ullrich2 Polina Kirichenko2 Ilia Kulikov2 Niraj K. Jha1 Mohammad Pezeshki2 Mark Ibrahim2

1Princeton University 2Meta FAIR

*Work done during an internship at Meta FAIR.

Paper Implementation BibTeX

When an agent fails, Self-Direction turns the failure into a small LoRA skill. The skill fixes the retry, joins a growing library, and is reused on related tasks later, while the base model stays frozen.

Terminal-Bench 2.1 task success 65.5% +13.8 pp vs. base model
Claw-Eval task success 73.3% +8.9 pp vs. base model
GAIA agent rollouts per task 2.1 65% fewer than pass@6, same 55.8% accuracy
Later failures that reused a stored skill 97.1% GAIA stream, 3-adapter library: no new training

Mean of three complete runs on Terminal-Bench 2.1 and Claw-Eval, with Qwen3.6-27B as the base model. Details in Results.

01 · The problem

Extra compute usually disappears with the episode

Modern agents can often correct a mistake and finish the task at hand, but the recovery is ephemeral: in a new session the agent is just as likely to make the same mistake again. As workflows grow longer, every extra step is another chance to slip, and rediscovering the same lesson over and over is costly and makes performance inconsistent. The goal is not only to fix the mistake now, but to keep the lesson so it is not repeated.

A mistake, up close · GAIA Level 1 What was the actual enrollment count of the clinical trial on H. pylori in acne vulgaris patients from Jan–May 2018, as listed on the NIH website?
Attempt 1 · base agent

47 of 50 steps are near-duplicate web searches. It keeps hitting a retired ClinicalTrials.gov endpoint, guesses trial IDs, and runs out of steps.

Answers 60 wrong
Self-reflection

The agent reads its own failed trajectory and the grader's verdict, writes 24 corrective examples ("use trial IDs for precise follow-ups", "strict step budget"), and trains a LoRA adapter on them.

Attempt 2 · base agent + adapter

Switches to Python and the ClinicalTrials.gov v2 API, finds trial NCT03411733 in a few steps, and confirms it on the study page.

Answers 90 correct

Without a persistent lesson, the next session starts from attempt 1 again. Example from the paper, Figure 15.

Why the usual fixes fall short

Retry or resample

Extra attempts recover the task now (+10.9 pp with pass@3 on GAIA), but the weights are untouched, so nothing carries over to the next session.

Keep lessons in context

Reflections and skill files stay outside the policy: they must be retrieved every time and compete for a finite context. On GAIA, the full learning history would take about 3.1M tokens, nearly 12 context windows; a compact Skills.md fits, but recovers fewer failures than in-weight lessons.

Write lessons into the weights

Full fine-tuning is expensive and can interfere with what the model already knows. Pooling every lesson into one LoRA also dilutes it: it recovers 16.7% of GAIA failures, against 27.7% with one adapter per failure.

Abstract

Intelligent agents should learn from their mistakes so experience compounds into continuous improvement. Yet today's agents recover from a mistake only temporarily: in-context reflections are easily forgotten across sessions and compete for limited context, while full-parameter fine-tuning is expensive and can interfere with existing capabilities. We introduce Self-Direction, a mechanism for agents to learn from mistakes that combines the benefits of temporary retries in-context and policy updates. When an agent fails a task, it reflects on its own trajectory in order to extract a lesson and immediately distills it into a lightweight low-rank adapter. By keeping the original weights frozen, Self-Direction preserves existing capabilities while only training 0.02% additional parameters for each adapter. The agent then saves these adapters into a persistent skill library, automatically routing them to solve related problems later.

Experiments with open agents based on the Qwen-family across open-ended research (GAIA), tool-use workflows (Claw-Eval), and terminal tasks (Terminal-Bench 2.1) show our method improves overall success rates by 8.9 to 17.0 percentage points and performs competitively with frontier systems. We find Self-Direction avoids repeating mistakes across future encounters of a task, nearly doubling the reliability of an agent's corrections and enables transfer of learned skills with just a handful of adapters generalizing to dozens of new tasks without additional training. Ultimately, this self-directed repair transforms everyday mistakes into lasting, reusable capabilities.

02 · Method

One loop: fail, reflect, distill, retry, reuse

The base model is never modified. Each lesson becomes a small adapter in a skill library, and a router decides whether a new failure needs a new skill or can reuse one it already has. The animation at the top walks through both paths.

  1. 1

    Attempt

    The frozen base agent attempts the task with its tools. When a verifier flags a failure, the task, the failure reason and the full trajectory are kept.

    wrong answer · no final answer · step or tool budget
  2. 2

    Reflect

    The same model reviews its own trajectory with the verifier's feedback and writes practice examples that teach the missing procedure: how to plan, use tools, verify and stop. It never writes the answer itself.

    up to 24 instruction → response examples
  3. 3

    Distill

    The examples train a small LoRA adapter while the base model stays frozen. The adapter joins the skill library, tagged with the failure that produced it.

    rank 16 · A from SVD of W, frozen · only B trained
  4. 4

    Retry & reuse

    The agent retries with the selected adapter and falls back to the clean base model if needed. Later failures can retrieve a stored adapter instead of training a new one.

    similarity ≥ τ → reuse · adapter first, then base

03 · Results

What the loop buys an agent

Self-Direction repairs mistakes across research, tool-use, coding and browser tasks, keeps those repairs on later encounters, and grows its skill library without disturbing what the agent already knows. Its skills also transfer to unseen tasks. Main experiments use Qwen3.6-27B; the paper's appendix repeats the loop on Qwen3.8-27B and Qwen3.5-9B.

3.1Paper §4.1

Repairs mistakes across task types

Over three runs, Self-Direction raises mean success on Claw-Eval from 64.4% to 73.3% and on Terminal-Bench 2.1 from 51.7% to 65.5%; on GAIA the gain is 17.0 points. It beats in-context reflections and a Skills.md file, matches the base agent's pass@6 on GAIA with only 2.10 rollouts per task, and narrows the gap to frontier agents. MiniWoB++ browser tasks improve as well, from 42% to 54%.

Terminal-Bench 2.151.7% → 65.5%
+13.8 pp
Claw-Eval64.4% → 73.3%
+8.9 pp
GAIA38.8% → 55.8%
+17.0 pp
Bar chart of Terminal-Bench 2.1 task success: base Qwen3.6-27B 51.7%, reflections in context 59.6%, Skills.md 56.2%, Self-Direction 65.5%. Pass@3 is 64.0% for the base agent and 74.2% for Self-Direction. Frontier references: GLM-5.1 58.6%, Gemini 3.1 Pro 65.6%, Opus 4.7 66.1%, GPT 5.6 Luna 75.7%.
Figure 1 (right). On Terminal-Bench 2.1, Self-Direction lifts performance from 51.7% to 65.5%, outperforming existing baselines and frontier models. Frontier numbers from tbench.ai.
Left: Claw-Eval task success: base 64.4%, reflections in context 70.0%, Skills.md 67.5%, Self-Direction 73.3%. Right: pass@3 rises from 73.9% to 80.7% and pass cubed from 51.6% to 64.6%, compared with frontier reference models.
Figure 4. Left: across three runs, Self-Direction improves mean success on Claw-Eval from 64.4% to 73.3% and outperforms the other baselines; error bars show one sample standard deviation. Right: it lets a 27B model outperform several larger frontier models on pass@3 and pass³.
GAIA task success by repair strategy: base 38.8%, base pass@3 49.7%, reflections in context 45.5%, Skills.md 53.3%, Self-Direction 55.8%.
Figure 2. Self-Direction outperforms alternative mistake-repair strategies on GAIA: 92 of 165 tasks (55.8%), ahead of retrying (pass@3), reflections in context, and reflections accumulated in Skills.md. It recovers 28 mistakes, a 17.0-point gain over the initial base run.
Accuracy against agent rollouts per task. Self-Direction reaches 55.8% on GAIA with 2.10 rollouts, matching base pass@6; 66.3% on Terminal-Bench 2.1 with 1.90 rollouts; and 75.8% on Claw-Eval with 1.54 rollouts, above base pass@3.
Figure 10 (appendix). Self-Direction reaches higher accuracy than base pass@k at a comparable number of task-environment rollouts. Rollouts exclude reflection, synthesis and adapter training.

3.2Paper §4.2

Learned skills prevent repeated mistakes

Across 185 prior mistakes, Self-Direction nearly doubles the fraction solved in all three encounters, from 9.7% to 18.9%, with both methods allowed up to two rollouts per encounter. Once it repairs a mistake, 76.9% of repairs still hold on a second encounter and 67.3% on a third, against 52.4% and 42.9% for base retrying. On Terminal-Bench 2.1 it lifts every run and raises pass³ from 34.8% to 57.3%.

solved in all 3 encountersbase retry 9.7%
18.9%
repairs kept on a 2nd encounterbase retry 52.4%
76.9%
Terminal-Bench 2.1 pass³base 34.8%
57.3%
Benchmark Encounters k Mistakes avoided (passk) Corrections preserved
Base retry Self-Direction Gain Base retry Self-Direction
GAIA29.9%21.8%+11.9 pp45.5%78.6%
36.9%19.8%+12.9 pp31.8%71.4%
Terminal-Bench 2.1213.6%20.5%+6.8 pp50.0%69.2%
313.6%15.9%+2.3 pp50.0%53.8%
Claw-Eval215.0%22.5%+7.5 pp75.0%81.8%
312.5%20.0%+7.5 pp62.5%72.7%
Total211.9%21.6%+9.7 pp52.4%76.9%
39.7%18.9%+9.2 pp42.9%67.3%
Table 1. Self-Direction prevents repaired mistakes from recurring. Mistakes avoided is the fraction of prior mistakes solved in every encounter through k; corrections preserved is the fraction of first-encounter successes that stay successful through k. Covers 101 GAIA, 44 Terminal-Bench 2.1 and 40 Claw-Eval prior mistakes.
Left: Terminal-Bench 2.1 success in three runs. The base agent scores 50.6%, 56.2% and 48.3%; Self-Direction scores 65.2%, 65.2% and 66.3%, gains of 14.6, 9.0 and 18.0 points. Right: pass@3 rises from 64.0% to 74.2% and pass cubed from 34.8% to 57.3%.
Figure 5. Self-Direction normalizes the variable performance of the base agent on Terminal-Bench 2.1. Left: base performance ranges from 48.3% to 56.2% across identical runs; Self-Direction holds every run at 65–66%. Right: pass@3 rises from 64.0% to 74.2% and pass³ from 34.8% to 57.3%. Frontier scores are shown for context; their harnesses differ.

3.3Paper §4.3

New skills without losing old ones

Every task first runs on the unchanged base model, and each skill is a separate frozen adapter, so learning cannot overwrite what the agent already knows. The router loads a stored adapter only when the new task is similar enough to the one it came from (cosine similarity ≥ τ). With one global τ = 0.55, a library holding every adapter from GAIA, Claw-Eval and Terminal-Bench 2.1 never sends a task to another benchmark's adapter. Sweeping τ from 0.2 to 0.6, task success holds for all but very permissive thresholds, below 0.30.

tasks misrouted at τ = 0.55all three benchmarks
0 / 370
task success preservedGAIA, Terminal-Bench 2.1
τ ≥ 0.30
parameters per adapterbase model frozen
0.02%
Benchmark pair Max. similarity Safe τ Cross-domain retrievals Capabilities preserved
GAIA – TB 2.10.5460.550 / 145
GAIA – Claw-Eval0.4150.420 / 141
TB 2.1 – Claw-Eval0.3060.310 / 84
All three, one global τ0.5460.550 / 370
Table 7 (appendix). Self-Direction learns new skills without disturbing existing ones. The safe threshold is the smallest two-decimal cutoff above the largest similarity observed across the pair. At this threshold no task retrieves an adapter from the other domain, in either direction, so an existing capability is never overwritten or misapplied.
Accuracy as the routing threshold varies from 0.20 to 0.45. For GAIA targets with Terminal-Bench and Claw-Eval adapters, and for Terminal-Bench targets with GAIA and Claw-Eval adapters, accuracy stays at the own-adapter baseline (20.8% and 22.7%) except at low thresholds below about 0.30, where it can rise or fall.
Figure 12 (appendix). Accuracy across router thresholds τ with a fixed library of cross-benchmark skills. Failed GAIA (left) and Terminal-Bench 2.1 (right) tasks are retried with the top-matched adapter from the other benchmarks. Existing capabilities are preserved except at thresholds below 0.30, where foreign adapters can help or hurt.

3.4Paper §4.4

Skills transfer to unseen tasks

In a shuffled stream of all 165 GAIA tasks that starts from an empty library, a library capped at three adapters (256 MB in total) raises task success from 36.4% to 52.7%. Skills learned on easier Level 1 GAIA tasks lift unseen Level 2 tasks from 40.7% to 51.2% (base pass@2: 43%), and three Claw-Eval operations adapters solve half of the failed workflow tasks, from 75.0% to 87.5%, without training on a single workflow task.

GAIA streamthree-adapter library
+16.3 pp
GAIA Level 1 → Level 228 adapters
+10.5 pp
Claw-Eval ops → workflow3 adapters
+12.5 pp
Generalization Skills learned from Library Unseen evaluation tasks Improvement
Tasks in a continual streamGAIA3 GAIA tasks3 adaptersRemaining GAIA tasks (n = 162)+16.3 pp36.4% → 52.7%
Harder tasksGAIAEasy Level 1 tasks28 adaptersHarder Level 2 tasks (n = 86)+10.47 pp40.7% → 51.2%
Tasks across categoriesClaw-EvalOperations tasks3 adaptersUnseen workflow tasks (n = 24)+12.5 pp75.0% → 87.5%
Table 3. Self-Direction transfers learned skills to unseen tasks. Stored skills generalize to new tasks in a continual deployment stream, to harder tasks, and to related task categories.
Left: adapter library size of 256, 383 and 895 MiB for 3, 6 and 12 trained adapters. Right: failures that avoided new training: 97.1%, 93.5% and 88.7%, with 18 of 102, 7 of 86 and 16 of 94 reuse wins.
Figure 11 (appendix). A compact skill library amortizes learning across a deployment-like GAIA stream: existing skills avoid new adapter training for 88.7–97.1% of initial failures. Bars show the library's storage; annotations report successful reuses.

04 · What gets learned

Skills are procedures, not answers

Reflection is told to teach the process and never to copy the answer, so a stored adapter carries a procedure that later tasks can reuse, across topics, modalities, difficulty levels and task categories.

Reuse in the shuffled GAIA streamPaper, Table 14

Source adapter · Level 3ORCID publication aggregation

Later target · Level 2Wikipedia revision counting

Traverse a complete remote history, handle pagination, apply a date constraint, and verify completeness before counting.

Initial answer 2610 → adapter answer 2732, correct.

Source adapter · Level 1BBC Earth video lookup

Later target · Level 1Exact quotation from another video

Identify the correct video and extract a localized fact using descriptions, timestamps, comments, or other supporting metadata.

The initial run exhausted its step budget; the adapter returned the exact quotation.

Source adapter · Level 1Spreadsheet aggregation

Later target · Level 2Statistics over values extracted from an image

Extract structured values, inspect intermediate data, compute with a tool, verify the calculation, and format only after validation.

Initial answer 2.404 → adapter answer 17.056, correct.

Source adapter · Level 2Caesar-cipher decoding

Later target · Level 1Text-grid reconstruction

Apply a text transformation, check linguistic coherence, and preserve exact capitalization, spacing, and output constraints.

The initial run recovered the meaning but broke the format; the adapter returned the exact required sentence.

Cross-category transfer on Claw-EvalPaper, Table 15

Operations skillScheduled-task management

Unseen workflowWeekly meeting tracking

Collect status across multiple records, identify missing or incomplete actions, and produce a structured progress report.

Grader score 0.413 → 0.946

Operations skillMonth-end reconciliation

Unseen workflowTask backtracking

Reconcile evidence across emails, task lists, and meeting notes; identify discrepancies and report status, blockers, and next actions.

Grader score 0.674 → 0.894

Operations skillScheduled-task management

Unseen workflowMulti-project progress reporting

Aggregate task status across several projects, identify risks, and organize the results into separate actionable reports.

Grader score 0.586 → 0.890

05 · Where it fits

Where the lesson is stored

The paper compares four ways to repair a mistake, which differ in where the lesson lives once the episode ends. Each card shows GAIA task success; the base agent alone scores 38.8%.

Not stored

Resampling

Spend extra test-time compute on independent samples from the unchanged model. Nothing is kept, even within the session.

49.7% GAIA, base pass@3

In the session's context

Reflect in-context

Put reflections or the failed trajectory into the prompt and retry. The lesson is forgotten once a new session starts.

45.5% GAIA

In a shared text file

Accumulate a Skills.md

Distill reflections across sessions into one markdown file loaded into context. It avoids context overflow but falls short of in-weight lessons.

53.3% GAIA

In the weights

Self-Direction

Distill each lesson into its own LoRA adapter, kept in a persistent, routable skill library. The base model stays frozen.

55.8% GAIA

Cite

BibTeX

@article{kansal2026selfdirection,
  title   = {Don't Forget What You Just Learned: Self-Directed Repair
             Turns Agent Mistakes Into Lasting Lessons},
  author  = {Kansal, Yuval and Ullrich, Karen and Kirichenko, Polina and
             Kulikov, Ilia and Jha, Niraj K. and Pezeshki, Mohammad and
             Ibrahim, Mark},
  journal = {arXiv preprint arXiv:2610.31846},
  year    = {2026}
}