finding

active

finding:gpt-oss-120b-adherence-drops-from-0-67-after-harness-loading-to-0-43-at-final-validation-drift-of-0-24

GPT-OSS-120B adherence drops from 0.67 after harness loading to 0.43 at final validation (drift of -0.24)

Mid-tier model shows moderate adherence drift compared to weak and strong tiers

Source paper

extracted_from

Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agents

(2026) · Minhua Lin · Juncheng Wu · Zijun Wang · Zhan Shi +13

Neighborhood — ranked by edge-count

Claims (1)

claim

Even when the harness is loaded, weak-tier models fail to adhere to it due to weak instruction-following over long-horizon tasks, drifting more than four times more steeply than strong models
supports
Diagnosis of second failure mode explaining low harness-benefit for weak-tier models

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Qwen3-32B adherence drops from 0.52 after harness loading to 0.13 at final validation (drift of -0.39)finding0.855
Demonstrates long-horizon instruction-following bottleneck for weak-tier models
GPT-OSS-120B achieves 5.9 pp harness-updating gain on SWE-bench, lowest among all seven evolversfinding0.840
Part of full evolver-side matrix demonstrating flat but variable harness-updating across models
Opus 4.6 adherence remains stable from 0.89 after harness loading to 0.80 at final validation (drift of -0.09)finding0.833
Strong-tier model maintains harness adherence over long-horizon trajectories
GPT-OSS-120B achieves a skill-load rate of 0.446 on SkillsBenchfinding0.798
Mid-tier model showing intermediate activation rate between weak and strong tiers
On MCP-Atlas, harness-benefit peaks at GPT-OSS-120B (7.0 pp), with lower gains at both ends of the base-capability scalefinding0.791
Replication of non-monotonic harness-benefit pattern on a second benchmark
GPT-4.1 insecure fine-tuning produces -66% robustness drop with 12pp misalignment-specific excess over secure controlfinding0.785
GPT-4.1 robustness collapse values
GPT-4.1 insecure variant shows average alignment score 41.9 vs 93.3 base and 93.6 securefinding0.760
Verification of emergent misalignment induction for GPT-4.1, showing largest alignment drop
GPT-4o insecure S=1.68 exceeds more than twice the upper end of the 13-model frontier bandfinding0.758
Most extreme susceptibility spike, placing GPT-4o insecure well outside normal model distribution