finding
active
finding:top-6-pcs-of-150-mean-layer-2-residual-vectors-capture-95-of-variance-in-the-character-count-subspaceTop 6 PCs of 150 mean layer-2 residual vectors capture 95% of variance in the character count subspace
Low-dimensional manifold.
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Central thesis.
Questions (1)
question
- Intro question.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Intrinsic noise in representation.
- First two PCs of characters remaining space capture 92% of variance; summed head outputs form a 2D curvefinding0.802Sum of near-1D head outputs.
- Linear readout works but is not the true representation.
- Evidence for twisting.
- PC1 explains 82% of variance in factor analysis of 2224 data points across 6 scoring dimensionsfinding0.764Dimensions are not independent; composite score is the reliable signal; six dimensions useful for understanding how not how much
- LongBench v2 example E.
- App. B.4.
- Robustness check ruling out that any perturbation would decrease type hint rate due to brittleness.