K values

In attention, key vectors that advertise 'where in the future should look here?'

Neighborhood — ranked by edge-count

paper

artifact

Janus Information Flow Transformers (Twitter thread, Sept 2025)
cites
Original thread by janus explaining transformer information highways and introspection capabilities, posted on X.

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

K values represent 'given current state, where in the future should look here?'claim0.817
Janus's interpretive claim about key vectors.
Q valuesconcept0.795
In attention, query vectors that ask 'where in the past should I look?' given the current state.
valueconcept0.747
Probability of sensory input expected by an agent, aligning value maximization with surprise minimization.
Value Iterationmethod0.721
A dynamic programming method for computing optimal value functions and policies in known MDPs.
V valuesconcept0.720
In attention, value vectors that carry the information future positions should receive.
k-means clusteringmethod0.709
Unsupervised feature-finding method using cluster centroid difference as feature direction
Epistemic Valueconcept0.704
Expected information gain about hidden states; drives curiosity and novelty-seeking; mutual information term in expected free energy.
negative valueconcept0.703
Negative of value, equated with free-energy and surprise.