concept
active
concept:4-layer-67m-parameter-model4-layer 67M-parameter model
The model (trained on The Pile) on which VPD is demonstrated to scale.
Neighborhood — ranked by edge-count
Findings (1)
finding
- Empirical demonstration of VPD on a mid-scale transformer, establishing feasibility.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- On calc (4+17)*2+7 Opus 4.5 J-lens shows 21, then 42, then 49 at successively later layers around layer 71.finding0.717Multi-step intermediates.
- SPD finds 102 W_in components and a rank-51 W_out component spanning all three layers in the 3-layer modelfinding0.716APD struggled beyond two layers.
- Claim that geometry-to-behavior correlates exist
- The layer approximately two-thirds through an LLM's transformer stack, reported to best predict human brain activity; identified as promising for consciousness indicators.
- Scale-dependent middle-layer representations.
- Fig. 5.
- Layers where anchoring weakens systematically due to representational drift.
- Concurrent work result showing emergent misalignment occurs in small models