artifact
pending-review
artifact:how-information-flows-through-transformersHow Information Flows Through Transformers
janus-information-flow-transformers-2025.mdFrontmatter (10 fields)
{
"author": null,
"context": "X/Twitter thread (Sept 10, 2025) proposing dual information highways in transformers: residual stream (vertical) and K/V stream (horizontal).",
"enrichment": {
"is_stale": true
},
"norm_label": "How Information Flows Through Transformers",
"source_url": null,
"graphify_id": "janus_information_flow_transformers_2025",
"source_file": "janus-information-flow-transformers-2025.md",
"imported_from": "/Users/antonborzov/Documents/Research.nosync/papers/extract_typed_out/janus-information-flow-transformers-2025/graph.json",
"extracted_type": "artifact",
"graphify_file_type": "artifact"
}Outgoing (10)
Cites (3)
- Anima Labs Conversation Part I(artifact)
- Causal graph formalism (Wolfram physics style)(framework)
- Koan Battery(method)
introduces (7)
- Information paths from A to B can exceed C(m+n, n) distinct routes, where m=position displacement and n=layer displacement.(finding)
- K/V Stream(concept)
- KV caching overcomes statelessness and provides mechanism for introspection of earlier token position computations.(claim)
- LLM introspection on internal computations is architecturally permitted; whether models leverage this is an empirical question.(claim)
- Q/K/V values function as information routing: Q queries past, K signals future attention, V carries selectively routed information.(claim)
- Residual Stream(concept)
- Transformer cognition operates via interference patterns across redundant paths, encoding nuanced state deltas similar to continuous human memory.(claim)
Mentions (1)
- papers-typed
janus-information-flow-transformers-2025.md