concept
active
concept:sparsity-reconstruction-tradeoffSparsity-reconstruction tradeoff
The balance between how sparse and how faithful a decomposition is; VPD achieves a better tradeoff than transcoders.
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Quantitative advantage claimed for VPD over a prior activation-decomposition method.
Concepts (1)
concept
- Sparsity-Fidelity Tradeoffrelated_toTraditional proxy evaluation criterion for SAEs.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Property that features activate on only a small fraction of inputs; enables compressed sensing and is what allows superposition to work
- Alternative sparsity notion proposed for predictive component.
- Classical literature on sparsity priors over admissible supports.
- Mechanistic argument.
- Playful and poetic amplification support.
- Empirical result demonstrating VPD's efficiency advantage in parameter decomposition.
- Known tension in dialogue generation where increasing diversity may reduce response relevancy
- SAEs assume sparsity (number of concepts) is time-invariant, drawn i.i.d. from a fixed distribution.claim0.737Corollary 4.1.1.