finding
active
finding:intrinsic-dimension-of-gemma-2-2b-and-llama-3-1-8b-activations-increases-steadily-with-sequence-position-while-a-stationary-surrogate-stays-nearly-constantIntrinsic dimension of Gemma-2-2B and Llama-3.1-8B activations increases steadily with sequence position, while a stationary surrogate stays nearly constant.
Fig. 2a,e.
Source paper
extracted_from(2025) · Lubana, Ekdeep Singh · Rager, Can · Hindupur, Sai Sumedh R. · Costa, Valerie +12
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (2)
claim
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Suggests architectural variations influence persona localization pattern
- Qualitatively different defense profile compared to Llama-3.1-8B
- Autocorrelation of LM activations varies with sequence position, unlike the stationary surrogate.finding0.791Fig. 2b,f.
- Interpretation of probe vs judge gap.
- Fig. 10.
- Shows the instruction effect, while shifting geometry, may not produce consistent generalization effects across model families.
- Corroborates role space findings using traits; shows PC1 also captures Assistant-ness in trait space
- Layer localization.