concept
active
concept:gelu-activation-function

GeLU Activation Function

The nonlinear activation function used in MLP layers; prevents the linearization approach used for attention layers from extending to MLP layers

Neighborhood — ranked by edge-count

Claims (1)

claim

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

  • Prior Anthropic approach to increasing neuron monosemanticity via activation function design; found to make some neurons more interpretable at cost of others
  • Activationsconcept0.686
    Internal representations of the model on which probes operate; the method uses activations to rank datapoints.
  • The conventional approach (e.g., SAEs, transcoders) of decomposing activations into interpretable features.
  • Activation Probingconcept0.665
    Technique of reading out model beliefs from internal activations before the final answer token is generated
  • Internals-based approach to which the logit monitor is complementary.
  • Seven binary classification tasks recast as natural-language yes/no questions.
  • Latent model activations when processing inputs framed from another agent's perspective
  • Using softmax to translate membrane potentials into firing rates, implementing lateral inhibition.