question
active
question:can-spd-be-scaled-to-larger-models-such-as-language-modelsCan SPD be scaled to larger models such as language models?
Open problem.
Source paper
extracted_from(2025) · Lucius Bushnaq · Dan Braun · Lee Sharkey
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Central claim.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Large language models develop surprisingly coherent yet often rigid internal preferences as they scalefinding0.788Mazeika et al. finding reinforcing the need for emptiness-based flexible value architectures
- Speculative α estimate.
- Survey of representation engineering methods cited as related work
- Scaling laws for dictionary learning are unknown and needed to assess feasibility on frontier models
- Core phenomenological argument.
- Limitation identified in Discussion; all models fine-tuned are <10B parameters
- We hypothesize that intervention efficiency can be scaled with multi-node and multi-GPU training as language models grow largerhypothesis0.759Future work hypothesis about scaling pyvene's computational efficiency for very large models
- Motivation for using sparsity-based dictionary learning on language models