Multi-layer Perceptron (MLP)

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Multi Layer Perceptronframework0.890
  Network with hidden layers capable of representing non-linearly separable functions, enabling deep model induction
• Multilayer Perceptronframework0.885
• MLP neuronsconcept0.826
  The sparse set of 28 neurons at layer 18 identified as responsible for Fourier feature computation across all cyclic tasks
• MLP layers are much harder to get traction on than attention layers; understanding them requires individually interpretable neurons which are rarely foundclaim0.789
  Key limitation of the paper's approach; MLP layers make up 2/3 of standard transformer parameters
• Perceptronmethod0.783
  Single-layer neural network that computes weighted sum of inputs; can only represent linearly separable functions
• The 28 MLP neurons at layer 18 can be partitioned into disjoint clusters each computing the sum for a Fourier feature with a different periodfinding0.774
  Structural finding showing modular organization within the sparse neuron set
• Perceptron Modelframework0.761
• A sparse set of 28 MLP neurons at layer 18 (~0.2% of MLP) are reused across all cyclic tasksfinding0.760
  Quantitative finding identifying the specific neurons responsible for generic addition