framework
active
framework:reverse-engineering-decomposition-description-validationReverse engineering (decomposition, description, validation)
Three-step approach to identify roles of network components.
Neighborhood — ranked by edge-count
Papers (1)
paper
- Open Problems in Mechanistic Interpretabilityintroducesmentions
Methods (4)
method
- Logit LensusesUnsupervised interpretability technique that projects activations through unembedding matrix; provides comparison point for NLA approach.
- Substitute activations to observe effects.
- General method for finding overcomplete sparse decompositions; the paper uses sparse autoencoders as an approximation
- Simplest method to describe causes of activation.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Load-bearing analogy defining the aspirational goal of mechanistic interpretability
- A class of methods that modify how models internally process representations; SOO fine-tuning fits within this framework
- Motivation for reverse engineering.
- Showed transformer representations predict brain representations in language areas; motivates Discussion about cortex as transformer.
- Key prior work on representation engineering that ReflCtrl directly extends
- Biological process demonstrating morphogenetic memory and goal-directed anatomical problem-solving; salamanders regrow exact organs and stop when correct structure is complete.
- Affiliation of several authors.
- The idea that the life of a building comes from the process of its creation, not from a preconceived design on paper.