Learning Skills from Experience — Inducing Reusable Workflows and Code Skills for AI Agents

Highlights: Daniel Fried’s lecture in Carnegie Mellon University’s CMU 11-768 AI Agents course (Fall 2026) examines how autonomous agents discover, verify, and maintain reusable skills across sequential web tasks. We walk through the representational trade-offs between natural language workflows and executable Python tools, automated verification and sandboxing, failure-driven strategy induction, and usage-based memory pruning.

• Task compression: Converting atomic browser interactions into executable code tools increases WebArena checkpoint completion from 41.3% to 80.2% while reducing average execution steps from 24.5 to 15.0.
• Over-retrieval penalties: Agent success on catalog tasks drops from 49.7% to 44.4% when retrieving four past experiences instead of one due to prompt clutter and distraction.
• Failure strategy induction: ReasoningBank abstracts high-level textual rules from failed navigation runs to improve performance, whereas replaying raw failed trajectories provides no benefit.
• Logarithmic pruning: TroVE evicts underused skills using a sublinear 0.5 * log10(n) usage cutoff over total tasks to prevent library bloat while protecting newly induced routines.
• Reinforcement learning for reuse: SAGE couples sequential tasks into joint training rollouts, awarding bonus rewards only when an induced skill is successfully reused in a downstream episode.

Tutorial Overview:

1. Skills as Reusable Task Chunks
2. Online Skill Induction Across Sequential Tasks
3. Inducing Parameterized Natural Language Workflows
4. Cross-Task Memory Compounding in Agent Workflows
5. Representational Trade-offs Between Text and Code
6. Inducing Executable Python Tools
7. Automated Linting, Verification, and Sandbox Testing
8. Task Compression and Execution Efficiency Gains
9. Interface Brittleness and Representational Control
10. PolySkill: Polymorphic Abstraction and Failure-Driven Learning
11. Learning from Failures and Strategy Induction
12. Evaluator Accuracy and Over-Retrieval Penalties
13. Memory Consolidation and Pruning via TroVE
14. Reinforcement Learning for Skill Inducers
15. Summary

---

1. Skills as Reusable Task Chunks

In short
– Agents package repeated task steps into modular skills to avoid re-planning.
– Low-level browser element identifiers break easily when webpage layouts change.
– Program induction frameworks synthesize executable libraries directly from raw execution traces.

1.1 Skills as reusable subroutines

Agents package recurring action sequences into modular skills to streamline execution across tasks.

• Running example: Consider an autonomous agent executing a routine web interaction, such as searching an online store catalog.
• Execution sequence: The agent fills fill('element145', {query}) and submits with click('element147').
• Shared chunk: Checking price ranges for wireless keyboards reuses the identical catalog search routine.
• Skill: A skill [reusable specification of a shared task segment] transfers across tasks.
• Brittle identifiers: Hardcoded DOM [Document Object Model] element IDs fail when dynamic page loads re-index elements.
• Cross-domain barrier: Navigating to an alternate website search interface breaks fixed action sequences entirely.

Fixed low-level actions fail whenever website layouts change. Abstracting subroutines preserves execution across interfaces.

1.2 Modularity in task decomposition

Decomposing interactions into modular chunks prevents agents from re-planning primitive steps for every new objective.

• Incremental composition: Agents assemble verified subroutines rather than synthesizing full interaction paths from scratch.
• Downstream targets: Shared search steps connect directly to price checks or cart operations.
• Stable foundation: Encapsulating verified capabilities shields planning from low-level interaction failures.

Fig 1. Modular building blocks let autonomous agents support varied downstream goals efficiently.

Modular decomposition prevents re-planning primitive steps for every goal. Verified routines create stable foundations for complex workflows.

1.3 Program induction foundations

Program induction [synthesizing code routines directly from raw execution traces] builds composable libraries.

• Voyager: Synthesizes executable Minecraft code skills where newly acquired functions call earlier routines recursively.
• DreamCoder: Abstracts shared subroutines across domain-specific grammars to compress the program library.
• Stitch: Accelerates library learning through fast symbolic compression over an execution corpus.
• LAPS: Directs search over program spaces using natural-language task descriptions.
• LILO: Pairs Stitch code compression with large language models to document induced components.

Fig 2. Program synthesis lineage progresses from symbolic library compression to language-guided induction.

Compressing repeated low-level behavior into reusable functions creates scalable procedural memory. Combining code execution with language reasoning scales agent capabilities.

---

2. Online Skill Induction Across Sequential Tasks

In short
– Agents discover reusable skills from sequential tasks to solve future problems without retraining.
– A three-phase lifecycle governs skill learning, execution, and active memory maintenance.

2.1 Foundations of Program Induction

Language agents build on program induction [The process of discovering and synthesizing reusable code functions or routines directly from raw execution traces.] to synthesize code.

• Voyager: induced executable skills to master Minecraft.
• DreamCoder: compressed programs by identifying recurring subroutines.
• Stitch: performs fast symbolic compression to discover shared programmatic structure.
• LAPS: guides library induction and program search using natural language task descriptions.
• LILO: combines language models with Stitch to synthesize and document compressed routines.

Agents build modular libraries instead of learning every complex procedure from scratch.

2.2 Sequential Online Skill Extraction

Agents receive tasks sequentially to incrementally construct an external skill [A reusable specification or subroutine of a shared task segment that an agent can apply across different tasks.] memory.

• Sequential arrival: the agent adds headphones to a wish list before querying keyboard prices.
• Induce after success: the agent extracts reusable components only after successfully finishing a task.
• Apply on later task: the agent retrieves stored routines to solve new goals.
• Continuous accumulation: shared experience compounds over time, avoiding redundant trial and error.

Fig 3. Agents extract skills after successful tasks and retrieve them for later problems.

Testing agents across sequential streams reveals how past interactions boost downstream task success.

2.3 The Three-Phase Skill Lifecycle

Daniel Fried structures skill management into three operational phases to govern execution.

1. Learn: executes judge model [An automated evaluation model that assesses an agent’s trajectory to decide whether an episode succeeded before admitting a skill.] checks, induces skills, and admits verified routines.
2. Use: retrieves relevant skills from storage, acts within the environment, and records outcomes.
3. Maintain: generalizes specific steps, repairs brittle routines, and retires obsolete code.

Fig 4. Skills move through continuous phases of learning, environment execution, and active maintenance.

Unchecked skill accumulation causes context clutter and distracts agents during planning.

2.4 Agent Workflow Memory

Zora Wang, Graham Neubig, and Daniel Fried designed Agent Workflow Memory to automate web navigation routines.

• Workflow induction: prompts models to convert traces into a workflow [A structured routine pairing a natural language subtask description with a sequence of parameterized action steps.].
• Variable abstraction: replaces specific search catalog items with parameters for future reuse.
• 2 action steps minimum: candidates must contain at least two steps to qualify for admission.
• Non-duplication rule: new candidates cannot duplicate existing workflows already stored in memory.
• Judge verification: a judge model confirms the trajectory solved the task before storage.

Strict admission filters guarantee the agent stores only clean, effective procedures.

---

3. Inducing Parameterized Natural Language Workflows

In short
– AWM converts successful trajectories into parameterized workflows instead of raw UI traces.
– Workflows generalize by replacing fixed element queries with descriptive template variables.
– Multi-stage filtering and judge models prevent flawed steps from poisoning memory.

3.1 Workflow extraction from trajectories

Agent Workflow Memory turns interaction traces into persistent skills [reusable cross-task subroutines].
1. Trajectory evaluation: verify episode success before extracting any candidate routines.
2. Sub-routine extraction: isolate recurring, non-redundant action sequences across multiple tasks.
3. Admission checks: a judge model [automated model evaluating trajectory success] validates final outcomes.

Fig 5. Three-stage filtering ensures only verified, non-redundant routines enter persistent agent memory.

Verifying episodes first keeps erroneous executions out of memory. This prevents flawed demonstrations from compounding over time.

3.2 Parameterized workflows for generalization

Agents abstract repetitive actions into workflows [routines pairing subtask descriptions with parameterized actions].
– Parameter abstraction: variable {term} replaces fixed text, like searching an online store catalog.
– Prompt injection: planners load step-by-step actions or text summaries based on context limits.
– 10% baseline accuracy: memoryless agents plateau while AWM maps walking routes and finds hotels.

Fig 6. Parameterized workflow templates enable agents to compound capabilities across complex sequential tasks.

# To find reviews for {term}, I will search for "{term}" in the reviews page
click("All reviews")
fill(757, "{term}")
click("Search")

Raw element identifiers break across different pages. Parameterized templates allow routines to generalize across tasks.

3.3 Induction prompts and admission rules

Structured few-shot prompts guide program induction [synthesizing reusable routines from execution traces].

Given a list of web navigation tasks, your task is to extract the common workflows.
Each given task contains a natural language instruction, and a series of actions to solve the task. You need to find the repetitive subset of actions across multiple tasks, and extract each of them out as a workflow.
Each workflow should be a commonly reused sub-routine of the tasks. Do not generate similar or overlapping workflows. Each workflow should have at least two steps. Represent the non-fixed elements (input text, button strings) with descriptive variable names as shown in the example.

• Two-step minimum: candidate workflows must contain at least two sequential actions.
• Overlap prevention: prompt rules forbid generating duplicate or structurally redundant routines.
• Descriptive parameters: templates replace fixed strings and element targets with descriptive variables.
• Judge state inspection: automated judges inspect final trajectory states before storing skills.

Unchecked extraction admits partial or failed steps. Strict admission protects memory against library poisoning.

---

4. Cross-Task Memory Compounding in Agent Workflows

In short
– Agent Workflow Memory retains verified subroutines to outperform memoryless agents on sequential tasks.
– Modular workflows enable solving composite requests by chaining previously discovered navigation routines.

4.1 Cumulative Retention in Sequential Tasks

Fried evaluates online learning agents using workflows (structured routines pairing subtask descriptions with action steps).

1. Extraction: the agent discovers reusable routines directly from successful task trajectories.
2. Storage: verified routines enter an external memory bank for persistent retention.
3. Retrieval: subsequent tasks fetch and adapt those existing routines during execution.
   – ~10% accuracy plateau: memoryless baselines stall while Agent Workflow Memory (AWM) drives continuous gains.
   – Amortized exploration: early workflow induction bootstraps performance and lowers search costs over extended sequences.

Retaining verified subtasks prevents agents from repeatedly re-solving recurring sub-problems from scratch.

4.2 Compounding Gains on Composite Workflows

Cross-task memory compounding allows agents to coordinate complex multi-step goals through modular chaining.

Attribute	Memoryless Baseline	Agent Workflow Memory
Accuracy	~10% plateau	Continuous improvement
Recurring subtasks	Re-solves from scratch	Reuses external bank
Composite goals	Fails routine chaining	Chains modular building blocks

• Composite goal failure: memoryless baselines fail when goals require chaining multiple previously discovered routines.
• Complex navigation request: finding a hotel near a location and showing the walking path.
• Compounding dynamics: structured textual memory transfers across sequential environments to widen performance margins.

Fig 7. AWM continuously improves map navigation success, whereas memoryless baselines plateau near 10%.

Modular routines serve as building blocks that unlock complex requests beyond memoryless capability.

---

5. Representational Trade-offs Between Text and Code

In short
– Executable code tools increase WebArena checkpoint completion rates from 41.3% to 80.2%.
– Text workflows provide runtime flexibility but incur severe per-step prompt reasoning overhead.
– Layered driver architectures and sandbox testing prevent cross-environment drift in code skills.

5.1 Flexibility Versus Determinism

Consider an autonomous agent executing a routine web interaction, such as searching an online store catalog.

Representation	Primary Advantage	Major Limitation
Workflow (structured routine pairing subtasks with actions)	Runtime interpretive flexibility	Manual per-step reasoning
Skill (reusable subroutine applied across different tasks)	Programmatic execution determinism	Rigid procedural execution

• Elements 1274, 473, 478: catalog search workflows record clicking, filling query terms, and submitting actions.
• Procedural rigidity: hardcoded code skills fail if the DOM (Document Object Model: webpage element structure) alters.
• Hierarchical composition: primitive functions nest inside higher-level routines to construct compound agent capabilities.
• Unit testability: standalone test suites verify functions independently of full web task trajectories.
• Code refactoring: parameter abstraction and modular refactoring clean and standardize agent skill libraries.
• Voyager framework: embodied agents demonstrated program induction (synthesizing reusable code from traces) in Minecraft.

1. Model inspects successful web navigation traces.
2. Program induction synthesizes modular functions like search_reviews and open_marketing_reviews.
3. Agent rewrites execution trajectories into high-level tool calls.
4. Framework validates candidate functions before long-term memory admission in Agent Skill Induction (ASI, Wang et al., COLM 2025).

def search_product(search_box_id: str, query: str):
    """Search for a product using the search box.
    Args:
        search_box_id: ID of the search input field
        query: Search query string to enter
    Returns:
        None
    Examples:
        search_product('595', 'sony bluetooth headphones')
    """
    click(search_box_id)
    fill(search_box_id, query)
    keyboard_press('Enter')

Text workflows require constant reasoning because models must interpret dynamic page states manually. Code functions eliminate reasoning overhead by enforcing deterministic execution sequences across identical interfaces.

5.2 Execution Overhead and Action Granularity

Atomic browser interactions create severe execution bottlenecks that degrade agent performance over long multi-turn horizons.

• Granularity bottleneck: processing high-dimensional screenshots and DOM trees consumes massive token budgets per step.
• 4 steps: dedicated tools achieve task compression (packaging low-level browser actions into high-level tools).
• 41.3% -> 59.5% -> 80.2%: checkpoint completion rates rise across five WebArena benchmark environments.
• 24.5 -> 20.6 -> 15.0: average task steps decrease from baseline to code tools.
• 5 benchmark environments: WebArena tests prove code skills drastically reduce latency and token consumption.
• External runtime: code execution relieves models from generating individual keystrokes and element clicks.

Fig 8. Packaging browser routines into tools shifts execution to runtimes, cutting multi-turn reasoning overhead.

Low-level actions fail over long horizons because visual token overhead exhausts limited context windows. Programmatic encapsulation succeeds because external runtimes handle keystrokes without repeated vision-language inference rounds.

5.3 Modality Strengths and Failure Modes

Skill representations balance flexibility against efficiency, requiring pre-admission testing to prevent interface-induced failures.

Memory Format	Advantage	Bottleneck	Verification
Raw Episodes	Preserves exact behavior	Excessive trace length	Trajectory replay
Text Skills	Flexible in-context advice	Zero step compression	Judge model
Code Skills	Composable execution speed	Cross-environment drift	Sandbox testing

• Pre-admission testing: a judge model (automated evaluation model assessing trajectory success) checks text outputs.
• Execution verification: sandbox testing (executing candidate code in isolated trial environments) confirms state correctness.
• Defect catching: static analysis / linting (automated inspection catching syntax defects) detects unused parameters.
• M366 imprint: pharmaceutical lookup linter flags unused “White” color parameter in identify_pill.
• Target.com failure: cross-environment drift (code breaking from interface changes) occurs when modal sidebars replace dropdowns.
• 3 architectural layers: PolySkill (Yu et al., ICLR 2026) separates base classes, site drivers, and dynamic dispatch (runtime driver routing).

1. Propose candidate skills for the target task in SkillWeaver (Zheng et al., 2025).
2. Practice candidate skills within the trial environment.
3. Evaluate action trajectories and visual screenshots using reward models.
4. Commit successful skills to memory or revise failing code.

candidates = propose_skill(task)
episode = practice(candidates, task)
if reward_model(episode.actions, episode.screenshots, task):
    memory.add(candidate)
else:
    candidate = revise(candidate, task)

identify_pill(page, imprint="M366", color="White")
+ if color:
+     await page.get_by_role("group", name="Color and shape (optional)") \
+               .get_by_role("combobox", name="Color (optional)").select_option(color)

Code skills outperform text prompts because automated testing catches defects before runtime deployment occurs. Layered architectures prevent cross-environment drift by isolating abstract intents from fragile site-specific selectors.

---

6. Inducing Executable Python Tools

In short
– Program induction extracts reusable Python functions directly from successful agent interaction histories.
– A three-step lifecycle evaluates, induces, and admits verified modular code into memory.
– Code skills enable hierarchical composition, unit testing, and significant context token savings.

6.1 Synthesizing Tools from Trajectories

The agent skill induction framework converts raw web interaction traces into verified Python functions.

1. Judge: a judge model (an automated evaluation model that assesses an agent’s trajectory to decide whether an episode succeeded before admitting a skill) evaluates task success.
2. Induce: program induction (the process of discovering and synthesizing reusable code functions or routines directly from raw execution traces) extracts reusable subroutines into modular code.
3. Admit: execution testing verifies the candidate code before adding it to memory.

• Voyager lineage: builds on Minecraft agents that synthesized nested functions for complex behaviors.
• Trajectory conversion: turns messy web interactions into clean, parameterized programs.
• Skill induction: creates a skill (a reusable specification or subroutine of a shared task segment that an agent can apply across different tasks).

Raw traces consume too much memory and cannot generalize. Parameterized functions make past actions reusable across new task instances.

6.2 Abstracting Actions into Code

Language models transform raw browser action sequences into callable routines with explicit arguments.

• Navigation helpers: abstract fixed click sequences into parameter-free helper functions.
• Parameterized tools: convert specific search terms and element clicks into flexible arguments.
• Abstracted rewrites: replace concrete browser episodes with calls to newly minted functions.
• Trace verification: runs the abstracted script in the environment to confirm target state.

def open_marketing_reviews():
    """Navigate to All Reviews under Marketing."""
    click("Marketing")
    click("All Reviews")

def search_reviews(search_box_id, search_button_id, search_term):
    """Search for reviews with a specific term."""
    fill(search_box_id, search_term)
    click(search_button_id)

open_marketing_reviews()
search_reviews('757', '704', "satisfied")
send_msg_to_user("2")

Executing rewritten code confirms whether the synthesized tools reach the target goal.

6.3 Operational Advantages of Code

Replacing primitive actions with executable code tools fundamentally reshapes autonomous agent operation.

• Hierarchical composition: nests simple functions into higher-level subroutines for multi-step workflows.
• Standalone testability: isolates and tests individual subroutines before deployment.
• Token savings: bypasses raw observation cycles to reduce context window usage.
• Online catalogs: consider an autonomous agent executing a routine web interaction, such as searching an online store catalog.

Fig 9. Executable code libraries provide hierarchical composition, modular testing, and reduced context consumption.

Calling high-level tools avoids re-reasoning over raw web observations at every step. This modular repository scales agent capabilities without exhausting context limits.

---

7. Automated Linting, Verification, and Sandbox Testing

In short
– Automated linting catches code syntax errors and unreferenced parameters before execution.
– Sandbox testing verifies executable routines and patches errors via Reflexion feedback loops.
– Abstract class interfaces decouple high-level workflows from fragile, site-specific webpage elements.

7.1 Skill Induction Admission Pipeline

Skill induction synthesizes raw agent trajectories into structured modular functions through an automated sequential admission pipeline.

• Raw trajectories: full traces of observations and low-level actions from agent environment interactions.
• Judge model: an automated evaluation model assessing episode success before admitting a skill.
• Program induction: synthesizing reusable code functions directly from successful execution traces.
• Parameterized output: functions containing typed arguments, docstrings, and usage examples for storage.

1. Execute the task episode in the target environment.
2. Evaluate episode completion using an automated judge model.
3. Induce parameterized code functions from successful interaction trajectories.
4. Verify and admit candidate routines into long-term memory.

Raw trajectories waste context space. Structured functions enable modular reuse across tasks.

7.2 Automated Linting and Parameter Checking

Static analysis inspects synthesized code functions to catch syntax errors and missing parameters prior to execution.

• Syntax validation: static analysis (automated code inspection catching defects before execution) catches syntax errors immediately.
• Iterative revision: failed candidate functions get reprompted with diagnostic compiler feedback for repair.
• Parameter checking: linters flag declared function arguments that the generated code body never references.
• Pill identification fix: a linter detected an unreferenced color parameter, prompting the addition of missing selectors.

candidates = propose_skill(task)
episode = practice(candidates, task)
if reward_model(episode.actions, episode.screenshots, task):
    memory.add(candidate)
else:
    candidate = revise(candidate, task)

identify_pill(page, imprint="M366", color="White")
+ if color:
+     await page.get_by_role("group", name="Color and shape (optional)") \
+               .get_by_role("combobox", name="Color (optional)").select_option(color)

Testing in live environments is expensive. Pre-execution linting repairs syntax flaws without wasting execution steps.

7.3 Unit Testing and Sandbox Execution

Sandbox testing executes candidate code skills in isolated target environments to verify functional correctness before admission.

• Reflexion-style loop: sandbox testing (isolated execution before saving) feeds error traces back to patch code.
• Task compression: packaging many low-level browser actions into a single high-level tool call.
• Latency reduction: agents delegate compound actions to scripts, regaining control only upon completion.
• 50 -> 4 steps: ASI framework solved address updates on a shopping site in 4 steps.

1. The model invokes the high-level function once with task-specific parameters.
2. The function script drives the browser through low-level DOM (Document Object Model) interactions.
3. The environment returns control only after completing the compound sub-routine.

Memory Type	Checkpoints Reached	Steps per Task
No memory	41.3%	24.5
Text skills	59.5%	20.6
Code skills	80.2%	15.0

Fig 10. Self-contained code skills execute compound browser interactions in a single tool call.

Atomic visual reasoning before every click consumes excessive tokens. Standalone scripts eliminate turn-by-turn round trips.

7.4 Cross-Site Generalization via Abstract Classes

Object-oriented design decouples abstract workflows from site-specific drivers to prevent cross-environment drift.

• Structural rigidity: hardcoded DOM element selectors fail when website layouts or element identifiers change.
• Catalog search failure: sorting store listings broke on target.com when dropdown menus became sidebar radio buttons.
• PolySkill framework: separates abstract method stubs from domain-specific primitive drivers for consistent execution.
• Subclass grounding: agents only write concrete interface methods when encountering novel web platforms.
• Dynamic dispatch: routing high-level abstract actions to site-specific browser drivers at runtime.

Fig 11. Decoupling abstract workflows from concrete DOM drivers enables cross-site skill reuse.

Writing new workflows for every website creates library bloat. Reusable classes keep code modular and adaptable.

---

8. Task Compression and Execution Efficiency Gains

In short
– Code skills compress multi-step interactions into compact tools to avoid failure caps.
– High-level execution removes repetitive screenshot parsing and chain-of-thought token generation.
– Programmatic abstractions boost benchmark checkpoints to 80.2% while cutting steps to 15.0.

8.1 Compressing Multi-Step Interactions

Fried compares baseline agents with code skill agents on an online store address update in Agent Skill Induction (ASI).

• Task compression (packaging actions into tools): converts atomic clicks and text inputs into composite tool calls.
• 50-step limit failure: atomic baseline clicks forms until reaching the cap.
• 4 steps to finish: code agent calls navigate_to_address_settings and update_address_details.

Packaging subroutines directly into programmatic tools bypasses lengthy action sequences that trigger execution limits.

8.2 Execution Step Reduction

Graphical user interface (GUI) agents face an efficiency bottleneck on multi-page web tasks.

• Visual token overhead: every extra step forces models to ingest dense screenshot tokens.
• Chain of thought delay: each action requires generating lengthy intermediate reasoning steps.
• Fewer inference calls: encapsulating browser interactions into functions eliminates redundant model round-trips.

Executable functions bypass repetitive perceptual processing. They directly eliminate unnecessary model round-trips.

8.3 Empirical Efficiency Gains

A controlled evaluation across five WebArena environments compares programmatic abstractions to baseline methods.

Agent Setup	Checkpoints Reached	Average Steps
No memory	41.3%	24.5
Text skills	59.5%	20.6
Code skills	80.2%	15.0

Fig 12. Code skills boost checkpoint success to 80.2% while slashing execution steps.

Deterministic subroutines run directly as tools instead of probabilistic prompt steps.

---

9. Interface Brittleness and Representational Control

In short
– Layout differences cause cross-environment drift when transferring skills across distinct websites.
– Skill representations trade contextual reasoning adaptability against browser execution efficiency.

9.1 The Site-Transfer Problem

Layout differences across websites break transferred skills.
– Cross-environment drift [The breaking of code skills when transferred to a new website because of changes in user interface design or element structures]: breaks skills transferred across sites.
– ASI benchmark: Wang et al. (COLM 2025) evaluate transfer on target.com.
– WebArena sort_listings skill: expects a dropdown menu to filter product catalogs.
– Target interface failure: renders a sidebar menu instead of a dropdown.

Procedural skills fail whenever target website structures diverge from induction environments.

9.2 Skill Representation Trade-Offs

Skill formats dictate trade-offs between adaptability and execution speed.
– Retrieved episode: preserves concrete execution traces but consumes heavy tokens and transfers poorly.
– Text skill: provides flexible natural language instructions for novel pages without execution speedups.
– Code skill: executes tool steps rapidly but breaks when DOM (Document Object Model) [The structural representation of webpage elements, forms, and buttons that agents interact with during web navigation] shifts.

Representation	Adaptability	Execution Speed	Primary Weakness
Retrieved episode	Low	Low	Token-heavy traces
Text skill	High	Low	Inefficient execution
Code skill	Low	High	DOM element fragility

Fig 13. Skill formats trade contextual reasoning adaptability against automated tool execution speed.

Skill representation balances contextual model inference against hardcoded programmatic routines.

---

10. PolySkill: Polymorphic Abstraction and Failure-Driven Learning

In short
– Decouple abstract shopping intents from site-specific browser code to prevent cross-site breakages.
– Extract high-level text strategies from failed executions instead of replaying broken low-level actions.
– Imperfect judge models provide sufficient guidance to expand skill libraries despite moderate label noise.

10.1 Abstract interfaces for generic intents

PolySkill (Yu et al., ICLR 2026) isolates web layout changes by separating task intents from browser execution.

• Base classes: specify generic shopping intents without relying on DOM (structural webpage element representations) layouts.
• Concrete subclasses: bind abstract signatures to site-specific browser interactions like custom dropdowns or sidebars.
• Dynamic dispatch: routes abstract actions to site-specific drivers at runtime, cutting token consumption.
• Catalog search example: high-level search routines remain unchanged when an online store catalog redesigns its pages.

Fig 14. Abstract base classes isolate site-specific code to prevent cross-site execution failures.

Decoupling task intent from execution shields plans when individual websites alter their page layouts.

10.2 Learning strategies from execution failures

Agents convert failed runs into high-level textual rules rather than replaying broken execution steps.

• Reflexion analysis: extracts query rules from pagination failures when searching an online store catalog.
• Search optimization: tightens keywords and applies sidebar category filters directly to prune catalog items.
• Trajectory replay: negative examples fail to improve raw traces or workflow (parameterized subtask routines) segments.
• Strategy extraction: natural language principles turn failed navigation episodes into effective avoidance policies.

Framework	Stored Representation	Negative Example Effect
Synapse	Full raw trajectories	No performance gain
AWM	Abstract workflow segments	Fails to help
ReasoningBank	Natural language strategies	Substantially improves success

### Induced Strategy: Search Query Optimization
- Avoid overly broad disjunctive keywords; tighten terms before browsing.
- Increase items displayed per page to minimize pagination overhead.
- Apply sidebar category filters directly to prune result sets.

Fig 15. ReasoningBank extracts high-level search optimization heuristics from failed navigation episodes.

Low-level traces repeat concrete errors, but abstract textual principles teach broad avoidance rules across tasks.

10.3 Judge calibration and ownership trade-offs

A judge model (an automated evaluation model assessing trajectory success) provides signal even under noisy labels.

• Graceful degradation: agent performance declines smoothly as judge accuracy falls from oracle to random chance.
• Authoring split: human engineers specify normative policies while autonomous agents induce empirical exploration tools.
• Verification roles: human inspection checks policy compliance while automated unit tests verify execution cheaply.
• Selection control: human operators handle rare edge cases while language models handle high-volume routine dispatch.
• Maintenance duties: human owners ensure organizational accountability while automated routines detect drift and evict code.

Autonomous skill libraries tolerate moderate evaluation errors, though human intervention safeguards critical organizational policies.

---

11. Learning from Failures and Strategy Induction

In short
– Abstracting failure traces into textual strategies prevents agents from repeating navigation traps.
– High-level strategic rules boost performance, whereas raw failure traces do not.

11.1 Failures in the skill lifecycle

An agent searching an online store catalog enters repetitive loops instead of refining queries.

• 5,578 results: searching “Bluetooth headphones Sony” displays 12 items per page across departments.
• Pagination loop: the agent repeatedly clicks “next page” until exhausting its step limit.
• Trajectory diagnosis: an LLM analyzes the failed run to induce corrective advice.

Discarding failures wastes diagnostic data. Trajectory diagnosis reveals behavioral traps before agents repeat them.

11.2 Extracting strategic guidelines from failures

ReasoningBank (Ouyang et al., ICLR 2026) distills high-level textual principles from failed trajectories.

• Query optimization: tighten vague search terms before browsing online store catalog listings.
• Display adjustment: increase displayed items per page to reduce navigation overhead.
• Category filtering: use sidebar category filters to narrow results without tedious pagination.

Textual feedback guides future behavior better than literal records of negative actions.

11.3 Impact across memory representations

Fried shows negative experiences benefit textual strategies while offering little to concrete traces.

• Synapse: concrete trajectory traces yield no meaningful improvement because failed action conditioning remains ambiguous.
• AWM: workflow (a structured routine pairing subtask descriptions with action steps) actions gain little benefit.
• ReasoningBank: high-level textual strategies achieve substantial performance gains on WebArena.

System	Memory representation	WebArena failure impact
Synapse	Concrete trajectory traces	No meaningful improvement
AWM	Semi-abstracted workflow actions	Little benefit
ReasoningBank	High-level textual strategies	Substantial performance gains

Fig 16. Failure memories improve task success only when abstracted into textual strategies.

Raw action traces leave failure causes ambiguous. High-level rules provide clear corrective instructions.

---

12. Evaluator Accuracy and Over-Retrieval Penalties

In short
– Task success stays stable near 49% across 70% to 90% judge accuracy.
– Excess demonstrations degrade agent success on catalog shopping tasks.

12.1 Impact of judge accuracy

Trajectory evaluation determines whether an autonomous agent admits an induced skill into memory.

• 47.6% to 52.4%: success scales from random 50% accuracy to 100% oracle verifiers.
• ~49% success rate: performance stays stable between 70% and 90% judge accuracy.
• ReasoningBank study: Ouyang et al. (ICLR 2026) evaluate a simulated judge model [An automated evaluation model that assesses an agent’s trajectory to decide whether an episode succeeded before admitting a skill.].
• Controlled flips: evaluator predictions flip at fixed synthetic error rates.

Imperfect verifiers filter memory candidates reliably. Flawless verification is unnecessary for effective skill induction.

12.2 Penalties from memory over-retrieval

The USE lifecycle phase executes two sequential steps before completing an action.

1. Retrieve candidate skills from memory.
2. Act in the environment conditioned on those demonstrations.

• 39.0% to 49.7%: success climbs on WebArena store catalog tasks with one retrieved experience.
• 49.7% to 44.4%: success drops across four experiences from over-retrieval [A performance drop caused by loading too many past experiences into the prompt context, which clutters reasoning and distracts the agent.].
• SkillsBench confirmation: human-authored skills document identical performance degradation.

Fig 17. Agent task success drops steadily when retrieving more than one memory experience.

Fried notes irrelevant memories clutter reasoning. Extra prompt tokens distract models during catalog navigation.

---

13. Memory Consolidation and Pruning via TroVE

In short
– Unchecked skill library growth causes selection bottlenecks that distract language models.
– TroVE evicts stagnant skills using a sublinear logarithmic threshold.
– Memory maintenance protects context efficiency and maintains high decision accuracy.

13.1 Tracking Skill Utilization and Identifying Redundancies

An active caching system evicts underused routines to prevent library bloat.

• Caching approach: operating memory like a cache drops stagnant skills systematically.
• Selection bottlenecks: unchecked growth introduces distractions from irrelevant tools during retrieval.
• Preventing interference: pruning curbs context inflation and stops cross-domain interference over streams.
• Decision accuracy: trimming unneeded tools preserves agent accuracy across lifelong tasks.

Fig 18. Tracking skill access frequency and pruning underused routines protects context efficiency and decision accuracy.

Unchecked repositories degrade agent reasoning during tool selection. Regular eviction keeps search spaces focused.

13.2 Usage-Based Pruning and Library Consolidation Thresholds

TroVE [a memory maintenance framework that automatically evicts underused skills using a logarithmic usage threshold] formalizes automated skill retirement.

• $\frac{1}{2} \cdot \log_{10}(n)$ threshold: drops routines used fewer times than this formula dictates.
• Total examples $n$: variable $n$ tracks all examples processed so far.
• Sublinear cutoff: logarithmic scaling grants newly induced skills an initial grace period.
• Transient pruning: rarely invoked routines get evicted systematically as total tasks increase.

Sublinear scaling prevents premature deletion of fresh skills while discarding long-term failures. Tool libraries remain compact over lifelong learning.

---

14. Reinforcement Learning for Skill Inducers

In short
– Standard prompt heuristics fail to optimize reusable skill induction across multi-task interactions directly.
– Rewarding downstream reuse trains agents to synthesize modular, generalizable code tools across tasks.
– Reinforcement learning regularizes memory size while balancing episodic storage against abstract code execution.

14.1 Multi-Task Interaction Feedback

Reinforcement learning directly optimizes non-differentiable skill [reusable task segment subroutine] induction pipelines across task sequences while regularizing memory growth.

• Ad hoc heuristics: Daniel Fried explains standard prompting merely guesses what skills might help later.
• Non-differentiable pipelines: reinforcement learning optimizes complex objectives including judge models [automated evaluation models checking trajectory success].
• Integrated execution: induction policies coordinate memory storage, retrieval, and code execution environments directly.
• Size regularization: explicit penalty terms restrict skill library growth to prevent task overfitting.

Standard prompt heuristics lack direct optimization signals. Reinforcement learning shapes policies using actual multi-task feedback.

14.2 Incentivizing Downstream Skill Reuse

Frameworks couple sequential tasks into joint training rollouts to reward skills that successfully aid future episodes.

1. Solve Task 1 and induce a candidate skill into shared memory.
2. Retrieve the candidate skill to solve Task 2.
3. Compute augmented reward conditioning directly on downstream utility:

r_augmented = r_1 + 1[both succeed] * 1[task 2 reused its skill]

• Joint rollouts: SAGE (Wang et al., ACL 2026) couples sequential tasks via shared memory.
• AgeMem framework: Yu et al. (ACL 2026) optimizes skill induction using multi-task execution streams.
• Downstream utility: reward bonuses require successful skill reuse in the second task.

Fig 19. Sequential task rollouts award bonuses only when downstream episodes reuse induced skills.

Single-task rewards fail to evaluate future utility. Rewarding reuse forces inducers to synthesize generalizable routines.

14.3 Policy Optimization with Reuse Rewards

Evaluating sequential task rollouts forces skill inducers to synthesize generalizable routines rather than overfit episodic artifacts.

• 55.4% -> 60.7%: AppWorld scenario completion increases using Qwen2.5-32B by rewarding successful skill reuse.
• Outcome baseline failure: rewarding single-task success does not measure broader utility across distinct user goals.
• Modular synthesis: policy optimization extracts generalizable workflows [structured routines pairing descriptions with action steps].

Fig 20. Rewarding skill reuse improves scenario completion over outcome-only baselines across distinct user goals.

Consider an autonomous agent searching an online store catalog. Joint rollouts penalize narrow shortcuts, teaching modular catalog search routines.

14.4 Long-Term Persistence in Agent Memory

Scaling agent memory over extended lifetimes exposes four essential trade-offs governing tool retention and execution.

Axis	Option A	Option B
Representation	Exact episodic traces	General abstract skills
Execution	Flexible language advice	Committed code functions
Library maintenance	Accumulate all experiences	Curate, merge, or retire

• Human oversight: human governance guides normative safety requirements and established operational policies.
• Autonomous induction: algorithmic discovery excels when learning through open-ended web environment interactions.
• Active curation: sustainable memory balances exploring new tools against reusing proven routines.

Unchecked accumulation degrades reasoning over time. Active curation ensures agents retain high-performing tools without context bloat.

---

15. Summary

In short
– End-to-end reinforcement learning optimizes skill induction across sequential task rollouts.
– SAGE rewards downstream reuse, while library curation sustains long-term competence.

15.1 Key Takeaways

Autonomous agents sustain competence by optimizing skill induction and curating memory stores.
– End-to-end RL: replaces heuristic prompting by directly optimizing skill (reusable task subroutine) induction across sequential task rollouts.
– SAGE framework: significantly boosts agent benchmark performance by coupling task rewards to successful downstream tool reuse.
– Library curation: balances procedural abstraction and continuous pruning to maintain skill stores over long operational horizons.
– Long-term competence: sustained by optimizing induction for downstream reuse and actively pruning memory stores.

---

Source

Lecture: Memory and Skills for Agents by Daniel Fried, Carnegie Mellon University.
Course: CMU 11-768 AI Agents (Fall 2026), Lecture 4, second half. The first half is covered in LLM_log #032.
Video: YouTube · Slides: PDF · Course page: cmu-agents.com

Further reading referenced in the lecture:

• Wang et al., Voyager (2023)
• Ellis et al., DreamCoder (2021)
• Bowers et al., Stitch (2023)
• Wong et al., LAPS (2021)
• Grand et al., LILO (2024)
• Wang et al., Agent Workflow Memory (ICLR 2025)
• Wang et al., Agent Skill Induction (COLM 2025)
• Zheng et al., SkillWeaver (2025)
• Yu et al., PolySkill (ICLR 2026)
• Ouyang et al., ReasoningBank (ICLR 2026)
• Wang et al., TroVE (ICML 2024)
• Not All Skills Help (2026)
• Wang et al., SAGE (ACL 2026)
• Yu et al., AgeMem (ACL 2026)

---

dataHacker.rs — LLM_log #033: Learning Skills from Experience — Inducing Reusable Workflows and Code Skills for AI Agents. By Vladimir Matic.