concept
active
concept:gradient-masking

Gradient masking

Adversarial robustness phenomenon by which a transcoder could cheat Jacobian matching.

Neighborhood — ranked by edge-count

Methods (1)

method
  • Jacobian matching
    associated_with
    Penalizing difference between transcoder and MLP Jacobians, computed cheaply via random vector products.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

  • causal maskingconcept0.820
    Attention restricted to previous tokens only, as in decoder-only models; leads to AR(ω)-like behaviour and no ordered phase
  • Frequent-task updates overwrite rare-task features.
  • Gradientsconcept0.801
    The property that qualities vary slowly, subtly, gradually across the extent of each living thing; gradients arise as natural responses to changing circumstances and create field-like character that points toward and establishes centers
  • Gradient methodmethod0.801
    Optimization technique that computes weight changes by following the gradient of an error function; contrasted with evolutionary stochastic search.
  • Gradient Descentmethod0.776
    Used for updating hidden state expectations; provides dynamical process theory testable against neuronal data
  • A structure-preserving transformation: using gradual change across space to soften and intensify transitions.
  • Baseline method against which probe-based ranking is compared; more computationally expensive.
  • Used by APD; poor approximations of causal importance.