concept
active
concept:dead-neuronsDead Neurons
Autoencoder neurons that fail to activate across any datapoints during training; addressed via neuron resampling
Neighborhood — ranked by edge-count
Methods (1)
method
- Neuron Resamplingassociated_withPeriodically reinitializing dead autoencoder neurons using high-loss data points to improve feature coverage
Concepts (1)
concept
- Feature Densityassociated_withFraction of training tokens on which a given feature has nonzero activation; used as proxy metric for autoencoder quality
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- MLP neurons and attention heads hypothesized to delete information from the residual stream by writing the negative of what they read
- The sparse set of 28 neurons at layer 18 identified as responsible for Fourier feature computation across all cyclic tasks
- Biologically plausible two-pool architecture from Krotov & Hopfield (2020) splitting self-attention into feature and memory neuron populations; used to interpret TEM-t place cells.
- Conventional brains where neurons maintain relatively fixed positions; a subset of collective intelligences.
- Neurons activated by the same pattern across a range of vision models, forming a common dictionary independently discovered by all models