finding
active
finding:at-1-density-the-pruned-vw-model-s-performance-is-significantly-compromisedAt 1% density the pruned VW model's performance is significantly compromised
Parameter count roughly matches original transformer.
Source paper
extracted_fromNicholas L. Turner · Jeffrey Wu · Joshua Batson
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Better bases may give sparsity.
Hypotheses (1)
hypothesis
- A better basis (e.g., factoring tokens into languages or parts of speech) may yield a sparse interpretable modelassociated_withSame model can be dense in one basis and sparse in another.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Fisher-effectiveness pruning at 30% density.
- Explicit pairwise weights.
- 15% density.
- Even restricted to most effective and helpful.
- Empirical demonstration of VPD on a mid-scale transformer, establishing feasibility.
- Rewriting of the 1L transformer into six families of virtual weights over tokens, positions, features, logits.
- Quantifies the misalignment-specific component of robustness collapse beyond generic fine-tuning costs
- Quantifies how much of the base model's diversity VS can recover compared to baseline prompting