claim
active
claim:llms-accumulate-in-context-evidence-sub-linearly-power-lawLLMs accumulate in-context evidence sub-linearly (power-law)
Justifies discount τ(N)=N^{-α}.
Source paper
extracted_from(2025) · Eric Bigelow · Daniel Wurgaft · YingQiao Wang · Noah D. Goodman +3
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (2)
finding
- ICL dynamics on Llama-3.1-8B are sigmoidal in N^{1-α}, fit by power-law-discounted likelihoodsupportsPrediction 1 confirmed.
- Fig. 2 baseline.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Establishes that the observed linear structure is not merely a representation of text probability
- Interpretive claim connecting scale to abstraction level in LLM representations
- Theoretical interpretation of antipodal alignment and misalignment phenomena in PCA visualizations
- Central empirical conclusion of the paper about the fundamental limits of truth directions.
- Interpretation of the layer-by-layer PCA visualizations showing linear structure emerging in early-middle layers
- The finding that interpretable concepts including character traits are encoded as linear directions in transformer residual streams
- Predictive hypothesis about Contemplative Architecture approach based on Petersen et al. 2025 work
- Localization result from patching experiments; identifies group (b) hidden states as the locus of truth representations