finding
active
finding:vw-model-contains-more-helpful-weights-than-parameters-in-original-transformerVW model contains more helpful weights than parameters in original transformer
Even restricted to most effective and helpful.
Source paper
extracted_fromNicholas L. Turner · Jeffrey Wu · Joshua Batson
Neighborhood — ranked by edge-count
Papers (1)
paper
Hypotheses (1)
hypothesis
- Same model can be dense in one basis and sparse in another.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Rewriting of the 1L transformer into six families of virtual weights over tokens, positions, features, logits.
- Explicit pairwise weights.
- Fisher-effectiveness pruning at 30% density.
- Antra's foundational claim about how introspection arises computationally rather than from memorised text.
- Parameter count roughly matches original transformer.
- Primary vision model architecture used in both vision-vision and cross-modal alignment experiments
- The model tends to reflect more when the question is difficult, and accuracy is generally lower for harder questionshypothesis0.725Hypothesis explaining negative correlation between reflection rate and accuracy without implying reflection is harmful
- Interpretation of the role of the direct path in multi-layer transformers; e.g. encoding that 'Barack' is often followed by 'Obama'