concept
active
concept:gelu-activation-function

GeLU Activation Function

The nonlinear activation function used in MLP layers; prevents the linearization approach used for attention layers from extending to MLP layers

Neighborhood — ranked by edge-count

Claims (1)

claim

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

  • Prior Anthropic approach to increasing neuron monosemanticity via activation function design; found to make some neurons more interpretable at cost of others
  • Activationsconcept0.686
    Internal representations of the model on which probes operate; the method uses activations to rank datapoints.
  • The conventional approach (e.g., SAEs, transcoders) of decomposing activations into interpretable features.
  • Activation Probingconcept0.665
    Technique of reading out model beliefs from internal activations before the final answer token is generated
  • Latent model activations when processing inputs framed from another agent's perspective
  • Using softmax to translate membrane potentials into firing rates, implementing lateral inhibition.
  • Experimental technique to induce bioelectric state changes and measure consequences for collective decision-making (morphogenesis, cancer, organ formation).
  • A failure mode where weak-tier models fail to invoke relevant harness artifacts (e.g., skills) during task solving