claim

active

claim:q-k-v-values-function-as-information-routing-q-queries-past-k-signals-future-attention-v-carries-selectively-routed-information

Q/K/V values function as information routing: Q queries past, K signals future attention, V carries selectively routed information.

Janus's interpretive model for how attention mechanisms enable deliberate information flow and selective routing.

Source paper

extracted_from

Janus Information Flow Transformers 2025

Neighborhood — ranked by edge-count

Communities (3)

community

Mechanistic interpretability & model evaluation
members_of
Spans attention head decomposition, benchmark awareness, and genomic pathogenicity prediction via neural models.
Mechanistic structure of transformer attention computations
members_of
Identifies distributed algorithms implemented across attention heads, with focus on causal masking limitations and emergent capabilities via activation manifold steering.
Distributed computation across attention heads
members_of
Studies how query, key, and value components decompose into specialized subfunctions across heads, enabling routing and token prediction behaviors.

Artifacts (1)

artifact

How Information Flows Through Transformers
introduces
X/Twitter thread (Sept 10, 2025) proposing dual information highways in transformers: residual stream (vertical) and K/V stream (horizontal).

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Q values represent 'given current state, where in the past should I look?'claim0.777
Janus's interpretive claim about query vectors.
A pair of query and key subcomponents distributed across attention heads performs syntax-boundary routingfinding0.755
VPD recovers an attention algorithm for routing across syntactic boundaries, distributed across heads.
V values represent 'given current state, what information should future positions that look here actually receive?'claim0.748
Janus's interpretive claim about value vectors.
Key, query, and value vectors are intermediary byproducts; W_OV and W_QK are the fundamental low-rank matrices describing attention head behaviorclaim0.748
Reframing observation: the canonical K/Q/V decomposition is computationally convenient but not the most interpretable representation
Information paths from A to B can exceed C(m+n, n) distinct routes, where m=position displacement and n=layer displacement.finding0.742
Quantifies extreme redundancy in transformer routing; supports claim that introspection and interference patterns are architecturally permitted.
attention-based signal routingconcept0.739
Mechanism by which attention heads detect injected perturbations and route information about them to the final token position
K values represent 'given current state, where in the future should look here?'claim0.736
Janus's interpretive claim about key vectors.
Each attention head has two largely independent computations: a QK circuit computing the attention pattern and an OV circuit computing the effect if attended toclaim0.734
Key decomposition enabling separate analysis of where attention goes and what it does