Rosetta Neurons

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Rosetta Neurons — individual neurons activated by the same patterns across a range of diverse vision models form a common dictionary independently discovered by all modelsfinding0.847
  Cited evidence that convergence extends to the neuron level, not just representational geometry
• MLP neuronsconcept0.766
  The sparse set of 28 neurons at layer 18 identified as responsible for Fourier feature computation across all cyclic tasks
• Feature Neurons and Memory Neurons Architectureframework0.721
  Biologically plausible two-pool architecture from Krotov & Hopfield (2020) splitting self-attention into feature and memory neuron populations; used to interpret TEM-t place cells.
• Memory Management Neurons/Headsconcept0.716
  MLP neurons and attention heads hypothesized to delete information from the residual stream by writing the negative of what they read
• Dead Neuronsconcept0.714
  Autoencoder neurons that fail to activate across any datapoints during training; addressed via neuron resampling
• No neuron found with Hebrew Unicode block in top dataset examples; most correlated neuron A/neurons/489 has correlation of only 0.1finding0.712
  Hebrew feature is effectively invisible in the neuron basis
• Curve detecting neurons found in every non-trivial vision model carefully examinedfinding0.709
  Empirical basis for treating curve detectors as a canonical example of meaningful, understandable features
• Neural Cellular Automataframework0.706
  Prior framework combining cellular automata with deep learning, extended by this work