The 28 MLP neurons at layer 18 can be partitioned into disjoint clusters each computing the sum for a Fourier feature with a different period

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• A sparse set of 28 MLP neurons at layer 18 (~0.2% of MLP) are reused across all cyclic tasksfinding0.870
  Quantitative finding identifying the specific neurons responsible for generic addition
• Fourier features with period 10 contribute to base-10 sum computation in the 28-neuron clusterfinding0.781
  One of the three base-10 Fourier periods identified in the sparse neuron set
• Disjoint neuron clusters for Fourier periodsconcept0.781
  The 28 identified neurons can be partitioned into disjoint clusters each computing a different Fourier period sum
• 2D projections of activations show clearly separable clusters for F0-F2 and A1 at layer 25, but increasingly entangled activations for F4-F5 and A2-A3.finding0.776
  Visual geometric evidence for the fundamental entanglement of true/false activations in harder tasks.
• Multi-layer Perceptron (MLP)method0.774
  Feed-forward neural network with hidden layers, capable of representing non-linearly separable functions.
• MLP layers are much harder to get traction on than attention layers; understanding them requires individually interpretable neurons which are rarely foundclaim0.773
  Key limitation of the paper's approach; MLP layers make up 2/3 of standard transformer parameters
• 512-neuron MLP continues to yield new features as autoencoder scales to 131,072 features (256× expansion)finding0.764
  Shows superposition enables many more features than neurons
• When and how can MLP neurons in transformers be individually interpreted, and what progress is needed to extend mechanistic interpretability to them?question0.761
  Major open problem identified in the paper; MLP layers constitute 2/3 of transformer parameters