question
active
question:can-we-transform-a-model-s-features-into-an-inexpensive-dense-signal-for-learning-open-ended-tasksCan we transform a model's features into an inexpensive, dense signal for learning open-ended tasks?
Motivating question.
Source paper
extracted_from(2026) · Aaditya Vikram Prasad · Connor Watts · Jack Merullo · Dhruvil Gala +3
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (1)
finding
- Topline result.
Claims (1)
claim
- Central thesis.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Core goal statement.
- As we make a model wider, do we just get more features, or the same features packed less densely?question0.788Fresh-traction question.
- Contrast with reward model non-identifiability.
- Width buys features in order of utility, so only larger models learn infrequent or complex tasksclaim0.784Interpretation of Theorem 3.
- A better basis (e.g., factoring tokens into languages or parts of speech) may yield a sparse interpretable modelhypothesis0.775Same model can be dense in one basis and sparse in another.
- Motivation for using sparsity-based dictionary learning on language models
- Clarifies what unsupervised learning does.
- Practical implication.