claim
active
claim:divergence-between-heatmaps-and-max-activating-examples-may-arise-because-logit-attribution-measures-a-feature-in-isolationDivergence between heatmaps and max-activating examples may arise because logit attribution measures a feature in isolation
Speculative explanation.
Source paper
extracted_fromNeighborhood — ranked by edge-count
Papers (1)
paper
Findings (1)
finding
Questions (1)
question
- Unclear reason stated by authors.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Input-output mapping per feature across ~100 prompts.
- Logit-minus-mean-logit loss was most effective for attribution, vs log-loss which was confusing to interpretfinding0.788Attribution loss selection.
- Design choice validated empirically.
- Feature 425104 heatmap promotes outputs above ~60 with little input dependence, e.g. promotes 62 given 90finding0.775Diverges from Hanna et al. heatmaps.
- Shows interpretability correlates with activation strength, most model effect comes from high activations
- Key geometry-to-behavior bridge finding in E3; robust to pooling choice, cosine vs. L2, and frozen external encoder
- Abstract parity feature.
- Motivates nested SAE.