Peters et al. 2018 Span Representation Method

Method concatenating boundary token vectors, their element-wise product, and difference to form span-level representations from (C)ARR.

Neighborhood — ranked by edge-count

thinker

Ganesh Jawahar
studies
Author whose methodology for span representations from BERT is adopted in this study.
Matthew Peters
introduces
Author of the span representation method (from ELMo paper) adopted in this study for computing Span Representations.

framework

Span Representation Analysis
implements
Framework for characterizing span-level information of sequences of representations, independent of any consciousness estimate; used as a comparison baseline.

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

span embedding analysismethod0.752
Extracting embeddings from instruction and example spans.
pyvene reproduces Meng et al. 2022 Figure 1 (factual association localization in GPT2-XL) in about 20 lines of codefinding0.719
Case Study I demonstrating pyvene can replicate a major interpretability result compactly
span embeddings extractionmethod0.719
Obtain instruction and example span embeddings at layer L* with chosen pooling.
Linear representationconcept0.707
The idea that features are encoded as directions in activation space.
We reproduce the results in Meng et al. (2022)'s Figure 1 of locating early sites and late sites of factual associations in GPT2-XL in about 20 lines of pyvene code.quote0.703
Load-bearing demonstration of pyvene's conciseness for complex replication tasks
Representation engineering for large-language models: Survey and research challenges (Bartoszcze et al., 2025)concept0.703
Survey of representation engineering methods cited as related work
Representation-based Pathconcept0.695
The path in activation space derived by fitting the representation manifold, used to steer along the geometric structure of internal representations.
In-Context Learning of Representations (Park et al. 2025)framework0.690
Reports phase-like breakpoints and geometry changes as context scales; UCCT provides measurable predictor