Noisy Simulation of Sparse Networks

Mechanism by which superposition works: small neural networks exploit sparsity to approximately simulate much larger sparse networks

Neighborhood — ranked by edge-count

framework

Superposition Hypothesis
associated_with
Core theoretical framework: neural networks represent more features than neurons by encoding features as directions in superposition

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Sparse interpretability in neural networksconcept0.763
VPD achieves sparse, interpretable parameter subcomponents with improved sparsity-reconstruction tradeoff.
Sparse circuitsconcept0.755
A goal in mechanistic interpretability to identify sparse computational subgraphs; VPD promotes sparse parameter circuits.
Greedy Algorithm for Network Coarse-Grainingmethod0.743
Method to aggregate nodes in complex networks to maximize EI, proposed by Klein & Hoel.
Sparse and smooth codingconcept0.733
Coding scheme where qualities are represented by few neurons with continuous similarity relations.
Sparse Dictionary Learningmethod0.728
General method for finding overcomplete sparse decompositions; the paper uses sparse autoencoders as an approximation
Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models (Marks et al., 2025)concept0.724
Cited as enabling precise behavioral control through SAE features, extending the same methodological line
Sparse Autoencoderframework0.723
Interpretability framework used to decompose layer-40 activations into sparse feature sets for studying emotional alignment and persistence
Neural networks, trained with different objectives on different data and modalities, are converging to a shared statistical model of reality in their representation spaces.quote0.722
The paper's central thesis statement, presented prominently after the abstract