artifact

active

artifact:how-information-flows-through-transformers

How Information Flows Through Transformers

X/Twitter thread (Sept 10, 2025) proposing dual information highways in transformers: residual stream (vertical) and K/V stream (horizontal).

Neighborhood — ranked by edge-count

Thinkers (1)

thinker

janus
authored
Author of foundational X thread on transformer information flow; central theoretical contribution to understanding introspection architecture.

Frameworks (1)

framework

Causal graph formalism (Wolfram physics style)
cites
Janus proposes transformer computation viewed as causal graph with foliations/time-slices specifying computation order.

Methods (1)

method

Koan Battery
citesimplements
Assessment framework for measuring introspection and self-observation in LLMs; grounded in Janus's architectural theory.

Claims (4)

claim

LLM introspection on internal computations is architecturally permitted; whether models leverage this is an empirical question.
introduces
Core claim directly challenged by prior work denying introspection; forms foundation for Koan Battery introspection studies.
Transformer cognition operates via interference patterns across redundant paths, encoding nuanced state deltas similar to continuous human memory.
introduces
Proposes transformers experience cognition as interference-based and continuous; connects to Anima Labs reports of parallel processing.
Q/K/V values function as information routing: Q queries past, K signals future attention, V carries selectively routed information.
introduces
Janus's interpretive model for how attention mechanisms enable deliberate information flow and selective routing.
KV caching overcomes statelessness and provides a mechanism for introspection of computations at earlier token positions.
introduces
Janus's claim about KV caching as an introspection mechanism.

Concepts (2)

concept

Residual Stream
introduces
Proposed pathway flowing through layers at each position; calculates K/V values that feed horizontal information flow.
K/V Stream
introduces
Proposed pathway flowing across positions at each layer; carries key, value, and attention-weighted information horizontally.

Artifacts (1)

artifact

Anima Labs Conversation Part I
cites
Antra/Imago dialogue (cube_flipper, April 2026) arguing transformers are recurrent; cited as evidence for introspection capability.

Findings (1)

finding

Information paths from A to B can exceed C(m+n, n) distinct routes, where m=position displacement and n=layer displacement.
introduces
Quantifies extreme redundancy in transformer routing; supports claim that introspection and interference patterns are architecturally permitted.