Harness Adherence Failure

A failure mode where even when harness artifacts are loaded, weak-tier models fail to follow their guidance faithfully

Neighborhood — ranked by edge-count

paper

concept

Harness-Benefit Capability
associated_with
The capability of a task-solving agent to benefit from updated harnesses during task solving
Harness-Following Rate
associated_with
The fraction of skill-loaded trajectories judged by an LLM judge as following the loaded skill's guidance
Long-Horizon Instruction Following
associated_with
The ability to sustain adherence to harness guidance over extended multi-turn trajectories, identified as a training target

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Harness Activation Failureconcept0.812
A failure mode where weak-tier models fail to invoke relevant harness artifacts (e.g., skills) during task solving
Harness-Updating Capabilityconcept0.747
The capability of an evolver model to produce useful persistent harness updates from execution evidence
Harness-Following Rate Measurementmethod0.734
LLM-judge pipeline measuring fraction of skill-loaded trajectories where agent follows loaded skill guidance, using Claude Sonnet 4.6 as judge
Agent Harnessconcept0.718
The external non-parametric context and infrastructure (prompts, skills, memories, tools) through which an LLM is deployed for task execution
Qwen3-32B adherence drops from 0.52 after harness loading to 0.13 at final validation (drift of -0.39)finding0.713
Demonstrates long-horizon instruction-following bottleneck for weak-tier models
Even when the harness is loaded, weak-tier models fail to adhere to it due to weak instruction-following over long-horizon tasks, drifting more than four times more steeply than strong modelsclaim0.713
Diagnosis of second failure mode explaining low harness-benefit for weak-tier models
Opus 4.6 adherence remains stable from 0.89 after harness loading to 0.80 at final validation (drift of -0.09)finding0.712
Strong-tier model maintains harness adherence over long-horizon trajectories
which models produce useful harness updates?question0.711
First open question the paper sets out to answer through evolver-side analysis