finding
active
finding:with-5-repeated-data-point-transcoder-learns-a-datapoint-feature-and-rotated-constituent-features-that-do-not-activate-on-pWith 5% repeated data point, transcoder learns a datapoint feature and rotated constituent features that do not activate on p
Demonstrates unfaithfulness in a simple setup.
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (2)
claim
- Takeaway of toy experiments.
- Memorization by transcoder not in model.
Questions (1)
question
- Why should we believe our transcoders learn the same mechanisms as the original underlying model?answered_byMotivating question.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Baseline toy experiment.
- Motivates Jacobian connection.
- Hyperparameter scan result.
- Cross-layer transcoder features form a mix of local, global and intermediate features by decoder normfinding0.758Preliminary decoder norm analysis.
- Motivating problem.
- Datapoint features form only if there is a repeated datapoint in training and sparsity regularization is appliedhypothesis0.748Two necessary conditions.
- Conceptual difference.
- Discussion.