question
active
question:how-should-we-decompose-networks-into-more-interpretable-constituent-partsHow should we decompose networks into more interpretable constituent parts?
Central open question A.1.1a.1.
Source paper
extracted_from(2025) · Lee Sharkey · Bilal I. Chughtai · Joshua Batson · Jack W. Lindsey +25
Neighborhood — ranked by edge-count
Papers (1)
paper
- Open Problems in Mechanistic Interpretabilityassociated_withmentions
Claims (1)
claim
- Networks do not naturally decompose into architectural components such as neurons, heads or layersgatesMotivates new decomposition.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Vision statement in the conclusion.
- Structured sparsity interpretability question.
- Core credit assignment question for distributed systems.
- Motivated by the finding that lexical entailment decomposes into word identities.
- Central claim from connectionist models: complex coordination emerges without centralized control or external teacher.
- Core claim of the paper: the right level of description for neural representations is geometric structure mirroring the world.
- Architectural requirement from machine learning.
- Closing statement.