finding
active
finding:pruning-85-of-vw-weights-by-fisher-effectiveness-costs-under-0-1-natsPruning 85% of VW weights by Fisher effectiveness costs under 0.1 nats
15% density.
Source paper
extracted_fromNicholas L. Turner · Jeffrey Wu · Joshua Batson
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Related work.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Fisher-effectiveness pruning at 30% density.
- Fisher effectiveness is roughly lognormal across 10 orders of magnitude; median is 10,000× below maxfinding0.746Versus VW approx normal, median one third of max.
- Parameter count roughly matches original transformer.
- ~27,000 point comparison.
- Even restricted to most effective and helpful.
- Baseline pruning by virtual weight magnitude.
- what α value provides the optimal balance between steering accuracy and response stability for different CV methods?question0.682Practical hyperparameter question motivating the injection strength sweep experiment
- Rewriting of the 1L transformer into six families of virtual weights over tokens, positions, features, logits.