concept
active
concept:mechanistic-faithfulnessMechanistic faithfulness
Idea that decomposed components must causally reflect the model's computation; enforced by adversarial ablation in VPD.
Neighborhood — ranked by edge-count
Papers (2)
paper
- A Toy Model of Mechanistic (Un)Faithfulnessintroducesmentions
Thinkers (1)
thinker
- Chris Olahstudies
Frameworks (1)
framework
- Refactoring of concerns into mechanistic faithfulness and simplicity.
Claims (1)
claim
- Methodological claim that the adversarial ablation approach ensures decomposed components causally correspond to computation.
Methods (1)
method
- Jacobian matchingaboutPenalizing difference between transcoder and MLP Jacobians, computed cheaply via random vector products.
Concepts (3)
concept
- Datapoint featureassociated_withTranscoder feature that activates on p and produces p, memorizing something the model did not.
- Simplicity (of transcoder explanations)associated_withSecond category of SAE/transcoder concerns, distinct from faithfulness.
- Error featuresassociated_withExtra feature representing unexplained error to make isomorphism exact.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Open issue in discussion.
- Graph-predicted indirect effects that do not exist in the underlying model.
- The condition that commitments are fulfilled.
- Takeaway of toy experiments.
- Three attributes for evaluating circuits.
- Authors' prioritization.
- Three desiderata for parameter components defining mechanisms.
- Empirical effect where intervening on one feature induces coherent shifts across multiple linguistic dimensions aligned with the target attribute.