claim
active
claim:unlike-fine-tuned-quirks-natural-behaviors-do-not-produce-anomalous-activation-tool-outputsUnlike fine-tuned quirks, natural behaviors do not produce anomalous activation tool outputs
Explains null result versus auditing games.
Source paper
extracted_from(2026) · Adam Karvonen · Euan Ong · Subhash Kantamneni · Samuel Marks
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (1)
finding
- No activation tool (AO, NLA, SAE) improves on transcript-only baseline for Gemma-3-27B-IT targetsupportsMain negative result; baseline AUROC ~81 with tools within ~1 point.
Hypotheses (1)
hypothesis
- Activations may be inherently incomplete for causal claims, requiring weight- or circuit-level accessassociated_withOne explanation for tool failure.
Questions (1)
question
- Unclear per Section 3.1.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- The core testable hypothesis driving the experimental design
- Cited from Wang et al. 2025a as reason SDF is preferred over demonstration fine-tuning for realistic model organisms.
- can we use the feature basis to detect when fine-tuning a model increases the likelihood of undesirable behaviors?question0.741Question about practical safety application of feature monitoring.
- If the relationship is present in activations, better tools (e.g. trained on these datasets) could surface ithypothesis0.740Alternative explanation.
- Contrast with simple gate MLPs.
- Possible explanation for why some concepts are more easily detected.
- Validates behavioral probe.
- Key interpretive conclusion from the dissociation between attempt rate and improvement rate in fine-tuning experiments