method
active
method:position-only-keys-queries-stimulus-only-values-factorizationPosition-Only Keys/Queries, Stimulus-Only Values Factorization
Key architectural modification restricting queries and keys to position encodings while values depend only on stimuli; extreme version of best-practice insight.
Neighborhood — ranked by edge-count
Papers (1)
paper
Frameworks (1)
framework
- TEM-Transformer (TEM-t)implementsThe transformer version directly analogous to TEM, introduced in this paper, offering dramatic performance improvements.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Mechanism for encoding sequence order in transformers; paper argues these should reflect learned structural representations rather than fixed sines/cosines.
- Attention mechanism in AI used in Transformers; also proposed for GWT implementations.
- Perturbation validation.
- Reframing observation: the canonical K/Q/V decomposition is computationally convenient but not the most interpretable representation
- Core slogan encapsulating the paradigm shift of VPD.
- The total Hamiltonian is equally consistent with any factorisation; establishes the boundary as a modelling choice not a physical fact
- Applied to 30-dim (layer, model) norm vectors to find shared structure across scale.
- When an intermediate key in a chain becomes unreachable, the model should re-associate values directly between the predecessor and successor.