finding

active

finding:otd-latent-activation-begins-declining-before-verbal-self-correction-appears-in-the-output-in-llama-3-3-70b

OTD latent activation begins declining before verbal self-correction appears in the output in Llama-3.3-70B

Temporal pattern consistent with internal monitoring process preceding explicit self-correction

Source paper

extracted_from

Endogenous Resistance to Activation Steering in Language Models

(2026) · Alex McKenzie · Keenan Pepper · Stijn Servaes · Martin Leitgab +5

Neighborhood — ranked by edge-count

Papers (1)

paper

Endogenous Resistance to Activation Steering in Language Models
introduces

Claims (2)

claim

Llama-3.3-70B exhibits internal consistency-checking mechanisms that operate during inference
supports
Central interpretive claim of the paper supported by causal ablation and activation evidence
The 26 differentially-activated OTD latents play a causally important role in enabling ESR in Llama-3.3-70B
supports
Causal interpretation of the ablation experiment results

Findings (1)

finding

Llama-3.3-70B corrected response scores 75/100 rather than 100 due to residual steering effects (Snell's law reference)
supports
Illustrative finding that ESR mitigates but does not fully eliminate steering influence

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

OTD latent ablation leaves mean first-attempt score unchanged (baseline 26.3, ablation 27.4) in Llama-3.3-70Bfinding0.828
Evidence that OTDs specifically support meta-cognitive monitoring rather than general response generation
Backtracking latents remain low during off-topic content and peak shortly after self-correction begins in Llama-3.3-70Bfinding0.828
Complementary temporal activation pattern suggesting distinct roles for OTD and backtracking latent classes
OTD latents fire 4.4× higher during off-topic content compared to baseline episodes without self-correctionfinding0.804
Quantitative characterization of OTD activation differential establishing their off-topic monitoring role
Ablating 26 OTD latents reduces multi-attempt rate by 25% (from 7.4% to 5.5%) in Llama-3.3-70Bfinding0.796
Primary causal evidence for dedicated internal consistency-checking circuits
Fine-tuning Llama-3.1-8B on self-correction examples increases multi-attempt rate proportionally with training data ratiofinding0.779
Shows behavioral pattern of self-correction is trainable in smaller models
Factual tasks F0-F3 reach near-perfect AUROC in early-to-mid layers of Llama-3.1-8B; arithmetic tasks A1-A3 emerge much later; counting tasks F4-F5 emerge late similar to arithmetic.finding0.777
Core empirical finding about layer-dependent truth direction emergence across task types.
Approximately half of the 26 OTD latents show near-zero or negative effect sizes, activating more during on-topic contentfinding0.768
Reveals that contrastive search yields a heterogeneous set, not all functioning as true off-topic detectors
26 candidate off-topic detector latents identified in Llama-3.3-70B via contrastive searchfinding0.762
Core mechanistic finding identifying specific SAE latents associated with ESR