concept
active
concept:feature-sparsity

Feature Sparsity

Property that features activate on only a small fraction of inputs; enables compressed sensing and is what allows superposition to work

Neighborhood — ranked by edge-count

Frameworks (1)

framework
  • Core theoretical framework: neural networks represent more features than neurons by encoding features as directions in superposition

Concepts (2)

concept
  • Superposition
    associated_with
    Phenomenon where models represent more features than dimensions via almost-orthogonal directions.
  • Average number of nonzero feature entries per input; primary measure of activation sparsity in the autoencoder

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.