framework
active
framework:virtual-weight-model-vw-modelVirtual weight model (VW model)
Rewriting of the 1L transformer into six families of virtual weights over tokens, positions, features, logits.
Neighborhood — ranked by edge-count
Papers (1)
paper
- Characterizing interference weights in a tiny language modelintroducesmentions
Methods (1)
method
- Used to express MLP in monosemantic features, 4096 features with JumpReLU.
Concepts (1)
concept
- Virtual WeightsimplementsImplicit weights directly connecting any pair of layers computed by multiplying output weights of one layer with input weights of another through the residual stream
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Even restricted to most effective and helpful.
- Explicit pairwise weights.
- Used to project features through QK weights and find aligned features.
- The phenomenon that separately trained models of the same architecture converge to the same basin and can be merged
- Secondary substrate demonstrating cross-modal applicability of manifold steering.
- Primary vision model architecture used in both vision-vision and cross-modal alignment experiments
- A computing layer above physical hardware; Sloman uses this as an analogy for mental processes not reducible to physics.