finding

active

finding:information-paths-from-a-to-b-can-exceed-c-m-n-n-distinct-routes-where-m-position-displacement-and-n-layer-displacement

Information paths from A to B can exceed C(m+n, n) distinct routes, where m=position displacement and n=layer displacement.

Quantifies extreme redundancy in transformer routing; supports claim that introspection and interference patterns are architecturally permitted.

Source paper

extracted_from

Janus Information Flow Transformers 2025

Neighborhood — ranked by edge-count

Claims (2)

claim

LLM introspection on internal computations is architecturally permitted; whether models leverage this is an empirical question.
supports
Core claim directly challenged by prior work denying introspection; forms foundation for Koan Battery introspection studies.
Transformer cognition operates via interference patterns across redundant paths, encoding nuanced state deltas similar to continuous human memory.
supports
Proposes transformers experience cognition as interference-based and continuous; connects to Anima Labs reports of parallel processing.

Communities (3)

community

Mechanistic interpretability & model evaluation
members_of
Spans attention head decomposition, benchmark awareness, and genomic pathogenicity prediction via neural models.
Mechanistic structure of transformer attention computations
members_of
Identifies distributed algorithms implemented across attention heads, with focus on causal masking limitations and emergent capabilities via activation manifold steering.
Empirical gaps in performance-communication alignment
members_of
Research identifying unexplored disconnects between stated sophistication/capability and measurable outcomes across domains, emphasizing need for direct empirical investigation.

Artifacts (1)

artifact

How Information Flows Through Transformers
introduces
X/Twitter thread (Sept 10, 2025) proposing dual information highways in transformers: residual stream (vertical) and K/V stream (horizontal).

Findings (1)

finding

Thought detection peaks at ~2/3 layer depth; intention checking peaks at ~1/2 layer depth.
supports
Lindsey (2026) differential layer performance explained by Janus's path combinatorics — different tasks use different path distributions.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Information from point A to B can travel through C(m+n, n) distinct paths, which quickly exceeds the number of atoms in the visible universe.claim0.838
Janus's mathematical claim about exponential path combinatorics in transformers.
Redundant information paths create interference patterns, so transformers likely experience memory and cognition as interferometric and continuous.claim0.753
Janus's claim linking path redundancy to interferometric phenomenology.
Representing non-linearly separable functions requires a network with multiple layers.claim0.750
Architectural requirement from machine learning.
Representation-based Pathconcept0.750
The path in activation space derived by fitting the representation manifold, used to steer along the geometric structure of internal representations.
The representation-based path and the behavior-based path in Llama-3.1 8B activation space trace out similar curves, demonstrating bidirectional geometry alignment.finding0.746
Key empirical result showing that optimizing for behavioral outputs and fitting representation geometry produce the same path in activation space.
No single layer is universally optimal for probing truth directions; different tasks peak at different layers.claim0.743
Argues against the single-layer analysis approach of prior work.
Q/K/V values function as information routing: Q queries past, K signals future attention, V carries selectively routed information.claim0.742
Janus's interpretive model for how attention mechanisms enable deliberate information flow and selective routing.
Information flow and increase must be understood relative to subsystems and their interaction with environment.claim0.738