concept
active
concept:interference-between-features

Interference between features

Cost of non-orthogonal feature embedding.

Neighborhood — ranked by edge-count

Papers (1)

paper

Thinkers (1)

thinker
  • Learning dynamics of linear networks used for mathematical understanding.

Concepts (2)

concept
  • When non-orthogonal features cause logistic regression to identify a suboptimal probe direction
  • Superposition
    associated_with
    Phenomenon where models represent more features than dimensions via almost-orthogonal directions.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

  • Frequent-task updates overwrite rare-task features.
  • Feature Sparsityconcept0.777
    Property that features activate on only a small fraction of inputs; enables compressed sensing and is what allows superposition to work
  • Error featuresconcept0.775
    Extra feature representing unexplained error to make isomorphism exact.
  • Feature splittingconcept0.771
    Phenomenon where a feature in a small SAE splits into multiple finer features in a larger SAE.
  • Patterns of which features activate together across tokens; preserved by covariance pooling but lost in mean pooling.
  • Shared featuresconcept0.764
    Features with similar decoder magnitude across both models.
  • Problems suggesting sparsity breaks down as interpretability proxy.
  • Feature Absorptionconcept0.760
    Phenomenon where sparsity yields gerrymandered latents for hierarchical concepts.