framework
active
framework:local-loss-landscape-decomposition

Local Loss Landscape Decomposition

Sparsely active parameter-space subnetworks reconstructing loss gradients.

Neighborhood — ranked by edge-count

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

  • Central quantity linking memorization and generalization.
  • The conventional approach (e.g., SAEs, transcoders) of decomposing activations into interpretable features.
  • Addressing disparity in loss magnitudes across tasks at the loss level
  • Loss on randomly masked subcomponents matching target outputs, with layerwise auxiliary.
  • Framework decomposing network parameters into sum of sparsely used vectors in parameter space.
  • Auxiliary training objective from Grant (2025) that constrains intervened representations to remain near natural distribution
  • Fitness Landscapeconcept0.687
    Kauffman's concept of the coupling between an evolving genotype and its fitness landscape as generating internal ordering tendencies in evolution
  • Theoretical distinction between representing token-context pairs as individual features (local) vs combining independent features (compositional)