concept
archived
concept:mechanistic-interpretabilityMechanistic Interpretability
Research field focused on understanding internal mechanisms and circuits within neural networks.
Neighborhood — ranked by edge-count
Papers (20)
paper
- The World Inside Neural Networksmentions
Thinkers (2)
thinker
- Chris Olahstudies
- Christopher Olahstudies
Frameworks (2)
framework
- Sparse AutoencoderimplementsInterpretability framework used to decompose layer-40 activations into sparse feature sets for studying emotional alignment and persistence
- The central mechanistic interpretability tool applied across all three EEG transformers to extract sparse feature dictionaries
Communities (1)
community
- Neural Steering Methodsmembers_of
Claims (3)
claim
- Core claim about why pernicious divergence undermines mechanistic conclusions
- Overall positioning.
- Extension of mechanistic interpretability findings to the metacognitive domain
Methods (3)
method
- Concept SteeringusesLatent intervention technique that manipulates sparse features to steer model predictions toward desired concepts.
- Spectral DecoderusesMethod that maps latent concept steering interventions back to EEG amplitude spectrum to obtain physiologically interpretable frequency signatures.
- Circuit tracingimplementsAnthropic technique revealing computational pathways in Claude.
Concepts (5)
concept
- Causal abstractionassociated_withextendsA framework the paper uses alongside feature geometry to deepen mechanistic understanding of LMs
- Manifold Steeringassociated_withCentral framework: steering neural networks by intervening along the curved manifold where a concept lives, rather than in straight lines through activation space.
- Universality HypothesisimplementsThe hypothesis that analogous features and circuits reliably form across different neural network models and tasks
- Activation ProbingimplementsTechnique of reading out model beliefs from internal activations before the final answer token is generated
- Microscope AIextendsUsing interpretability for scientific knowledge discovery from trained models.
Artifacts (4)
artifact
- Large Language Models Report Subjective Experience Under Self-Referential Processingassociated_withmentionsKey paper finding structured first-person descriptions in LLMs claiming awareness or subjective experience during self-referential processing.
- SilicoaboutAutonomous LLM agent that produced the research and initial draft under human direction.
- GarconaboutA software library created by Nelson Elhage for accessing model activations and parameters regardless of scale; crucial infrastructure for the paper's analysis
- PySvelteaboutA library released by the authors making it convenient to use interactive visualizations from Python; used for exploring model activations
Institutes (2)
institute
Hypotheses (1)
hypothesis
- First falsifiable prediction of the thesis, testable in AI systems via mechanistic interpretability
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Explaining activations without understanding mechanism.
- Functionally significant directions in the residual stream that organize activations in mechanistic interpretability
- Goodfire's stance drawn from experience with Evo and R1.
- The capability to explain model predictions; a central theme of the paper, with disruption profiles as vehicle.
- An inference path across features that constitutes a computational subgraph in a trained model
- Socio-philosophical open question.
- Method using large language models (Claude) to generate and test explanations of features at scale
- Model organism choice.