finding

active

finding:fine-tuning-llama-3-1-8b-on-self-correction-examples-increases-multi-attempt-rate-proportionally-with-training-data-ratio

Fine-tuning Llama-3.1-8B on self-correction examples increases multi-attempt rate proportionally with training data ratio

Shows behavioral pattern of self-correction is trainable in smaller models

Source paper

extracted_from

Endogenous Resistance to Activation Steering in Language Models

(2026) · Alex McKenzie · Keenan Pepper · Stijn Servaes · Martin Leitgab +5

Neighborhood — ranked by edge-count

Papers (1)

paper

Endogenous Resistance to Activation Steering in Language Models
introduces

Claims (1)

claim

Fine-tuning induces the behavioral pattern of self-correction but does not improve the underlying ability to correct effectively
supports
Key interpretive conclusion from the dissociation between attempt rate and improvement rate in fine-tuning experiments

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Llama-3.3-70B shows multi-attempt rate of 7.4% vs. ≤1.2% for all other models testedfinding0.848
Supporting finding showing ESR is driven by both higher multi-attempt rates and comparable improvement rates
Multi-attempt improvement rate peaks at 83% around -1.0σ below threshold in Llama-3.3-70Bfinding0.847
Shows slightly weaker steering allows more successful corrections, characterizing optimal ESR conditions
Llama-3.3-70B corrected response scores 75/100 rather than 100 due to residual steering effects (Snell's law reference)finding0.835
Illustrative finding that ESR mitigates but does not fully eliminate steering influence
Meta-prompting increases Llama-3.3-70B multi-attempt rate 4.3× (from 7.4% to 31.7%)finding0.826
Demonstrates ESR can be deliberately enhanced through prompting in the largest model
Llama 3.1 405B shows 14% compliance gap in minimal helpful-only replication but smaller Llama and Mistral models show no gapfinding0.817
Replication across open-weight models supports scale-emergence finding
The difficulty boundary for truth directions replicates across all four tested models (Llama-3.2-3B, Llama-3.1-8B, Gemma-2-2b, Gemma-2-9b); generalization to F3-F5 remains consistently low regardless of model size or family.finding0.812
Establishes generalizability of the core difficulty-boundary finding across model families.
Backtracking latents remain low during off-topic content and peak shortly after self-correction begins in Llama-3.3-70Bfinding0.806
Complementary temporal activation pattern suggesting distinct roles for OTD and backtracking latent classes
LLaMA-2-70B and 13B probes generalize better across datasets than 7B probes across all training sets and probe typesfinding0.804
Larger models linearly represent more general concepts including truth