artifact
active
artifact:semantic-anchoring-in-llms-thresholds-transfer-and-geometric-correlatesSemantic Anchoring in LLMs: Thresholds, Transfer, and Geometric Correlates
Main paper presenting UCCT and semantic anchoring framework.
Neighborhood — ranked by edge-count
Thinkers (46)
thinker
- Jason WeicitesmentionsEmergent abilities of LLMs.
- Trenton BrickencitesmentionsToy models of superposition.
- Catherine Olssoncitesmentions
- Stanislas DehaenecitesmentionsCited for global workspace theory and consciousness models.
- Johannes von OswaldcitesmentionsTransformers learn in-context by gradient descent.
- Anne M. TreismancitesmentionsCited for feature integration theory of attention.
- Core Francisco ParkcitesmentionsCompetition dynamics in ICL and representation geometry.
- Daniel KahnemancitesmentionsCited for dual-process theory.
- Tom B. BrowncitesmentionsLead author of GPT-3 paper, demonstrating few-shot learning.
- Bernard J. BaarscitesmentionsCited for global workspace theory.
- Guangyu HongcitesmentionsMixtures of in-context learners (MOICL).
- Jesse HooglandcitesmentionsDevelopmental landscape of in-context learning.
- Jinwu HucitesmentionsTest-time learning for LLMs.
- Michael I. PosnercitesmentionsCited for attentional networks.
- Rylan SchaeffercitesmentionsAre emergent abilities a mirage?
- Sang Michael XiecitesmentionsBayesian explanation of ICL.
- Sewon MincitesmentionsRethinking the role of demonstrations in ICL.
- Siyin WangcitesmentionsBayesian example selection for ICL.
- Zihang DaicitesmentionsMeta-learning analogy for ICL.
- Edward Y. Changauthored
- Ethan Y. Changauthored
- Christopher M. BishopmentionsMachine learning textbook author.
- Rabiner 1989citesAuthor of a foundational tutorial on hidden Markov models.
- Zeyneb N. Kayaauthored
+22 more
Frameworks (11)
framework
- Global workspace theorycitesmentionsTheory of consciousness involving a global workspace for information.
- Dual-process theorycitesmentionsDistinguishes fast pattern completion from deliberative control, used as analogy in paper.
- Unified Contextual Control Theory (UCCT)introducesA theory that pretrained latent patterns are bound to task targets via external semantic anchors; formalized by anchoring strength S.
- Formalizes regime shifts between retrieval-like and inference-like ICL; UCCT complements with when-predictor
- Prior framework explaining ICL as inference over task structure; UCCT adopts and extends the Bayesian lens
- Studies how inputs are gated in attention, cited as analogy.
- Reports phase-like breakpoints and geometry changes as context scales; UCCT provides measurable predictor
- Selection/weighting strategy for ICL demonstrations; in UCCT terms alters context posterior
- Views ICL as a form of meta-learning; UCCT sits alongside this account
- Selection strategy that adapts which demonstrations carry signal; in UCCT terms increases effective ρd
- Views transformers as performing implicit gradient descent; UCCT complements this mechanistic account
Methods (15)
method
- Quantitative study correlating layer-wise anchoring geometry (S_max, AUS_N) with behavioral thresholds θ50
- Quantitative study varying representational familiarity via numeral bases B10/B8/B9 at fixed computational complexity
- whitening and z-scoring procedureintroducesCalibration protocol: whiten embeddings on dev pool, z-score ρd and dr per layer.
- layer-wise trajectory analysisintroducesComputing per-layer S(ℓ) to summarize geometry.
- per-dev z-scalingintroducesStandardizing ρd and dr using dev-set means and stds to form dimensionless components of S.
- whitening and z-scoring protocolintroducesStandardization of ρd, dr, and log k on dev set for computing S.
- Compute per-layer S(ℓ) = ρ̃d(ℓ) - d̃r(ℓ) - log k after whitening and standardization.
- logistic fitting for shot thresholdsintroducesFit a sigmoid to accuracy vs. k to estimate k50 and phase width.
- Logistic surrogate fittingintroducesFitting a logistic function to success probability as a function of S or shot count to estimate midpoints and widths.
- logistic surrogate modelintroducesSigmoid fit linking S to success probability.
- Per-dev z-scoringintroducesStandardization of ρd and dr components using development-set mean and standard deviation.
- Retrieving external content to augment prompts.
- Whitening of span embeddingsintroducesPreprocessing step that uses dev-set covariance to standardize embedding scales before computing ρd and dr.
- Qualitative experiment showing coherent anchors can rebind strong priors across text and vision modalities
- Geometry summaries (Sbmax, AUSN)introducesPeak anchoring (Sbmax) and normalized area under the S(ℓ) curve (AUSN) used to summarize trajectory.
Artifacts (28)
artifact
- Reproducibility package released with the paper.
- Documented threshold-like emergent behaviors.
- Cognitive framing of access and broadcasting.
- Textbook reference for mixture models and sufficiency.
- Superposition and sparse feature structure.
- Foundational ICL paper cited for few-shot capability and anchoring concept.
- ARC dataset cited for evaluation tasks.
- Meta-learning analogy for ICL.
- Global workspace theory cited as analogy for selective access.
- Selection/weighting of demonstrations.
- Stagewise geometry preceding behavioral milestones.
- Test-time adaptation techniques.
- Code task dataset.
- Dual-process theory cited as cognitive analogy.
- Logical inference dataset.
- Showed format dominates label use in ICL.
- Mechanistic work uncovering induction heads.
- Formalized regime shifts between retrieval-like and inference-like ICL.
- Geometry changes under context scaling.
- Attentional networks theory.
- HMM reference for sufficiency assumption.
- Examined emergence vs. metric sharpness.
- Commonsense reasoning dataset.
- Classic attention gating work.
+4 more
Concepts (19)
concept
- anchoring strength SintroducesComposite score S = ρd − dr − log k predicting anchoring success.
- Test-time adaptation from prompt or retrieved context with no parameter updates.
- semantic anchoringintroducesThe central idea that external structure binds latent patterns to desired targets.
- Fine-tuningaboutParameter updates that reduce mismatch dr; another anchoring variant in UCCT.
- Anchoring strength S = ρd - dr - log kintroducesThe calibrated score measuring how effectively anchors bind target patterns; a predictive correlate of success.
- shot midpoint k50introducesNumber of in-context exemplars to reach 50% accuracy in E2.
- Sharp performance changes when S crosses a critical value.
- internal threshold θ50introducesFew-shot midpoint in E3's geometric analysis.
- Logistic success surrogateintroducesPhenomenological fit P(success)=σ(αS+β) used to summarize sharpness and midpoints.
- Normalized area under S (AUSN)introducesAverage of per-layer S(ℓ) scores, summarizing the breadth of anchoring trajectory.
- cohesion ρdintroducesWithin-cluster tightness of target pattern representations.
- Mismatch drintroducesDistance between prior knowledge centroid and target pattern centroid, e.g., 1 - cos(eprior, eT).
- peak anchoring SbmaxintroducesMaximum layer-wise anchoring score across layers.
- Observation that anchoring effects appear across text and vision modalities.
- Emergent Abilities of LLMsmentionsPrior work documenting abrupt capability changes under scale; UCCT provides a measurable predictor for when they occur
- task-dependent threshold ScintroducesCritical anchoring strength above which performance flips sharply.
- Threshold ScintroducesValue of S above which performance sharply increases; varies by model, layer, and task.
- anchor budget kintroducesNumber of few-shot exemplars provided.
- Phenomenon documented by Park et al. 2024/2025 that UCCT complements by providing a when-predictor
Claims (14)
claim
- UCCT strictly generalizes ICL and reads retrieval-augmented generation and fine-tuning as the same anchoring process acting on one measurable score SintroducessupportsAuthors' central interpretive claim about the scope of their theory
- Main interpretation of E3.
- E2 main interpretive claim.
- Scope-limiting claim clarifying UCCT's interpretation of what anchoring does
- A central claim about the operational value of S.
- Predictive practical utility claim.
- Cross-domain anchoring claim.
- Interpretation of abrupt behavior changes.
- Layer-wise geometry shows early dip, mid-layer alignment, and late standardization across taskssupportsQualitative pattern from E3.
- Assertion about the nature of prompt engineering.
- Clarifies nature of S.
- Shot midpoints follow k50 ∝ dr/ρd; higher cohesion and lower mismatch yield fewer required examplesintroducesCore quantitative prediction of UCCT validated by E2 threshold ordering
- Falsifiability claim.
- Interpretation of E3 layer-wise results; motivates targeted UCCT interventions at layers 8-12
Hypotheses (3)
hypothesis
- Core testable hypothesis of UCCT about the nature of performance transitions under anchoring
- E3 prediction that internal geometry provides a bridge to behavioral thresholds
- Hypothesis: Shot midpoint ordering k50(B10) < k50(B8) ≈ k50(B9) follows pretraining exposure densityintroducesE2 prediction that bases with higher pretraining exposure require fewer shots to cross threshold
Events (3)
event
- Qualitative tests of anchor rebinding of strong priors in text and vision.
- Varying representational familiarity at fixed complexity to test k50 ordering and transfer.
- Layer-wise geometry analysis linking Sbmax/AUSN to θ50.
Datasets (2)
dataset
- Synthetic arithmetic datasets for base-10, base-8, base-9 two-digit addition with train and test splits.
- Set of 25 prompts spanning commonsense, logic, science, arithmetic, code tasks used for layer-wise geometry overlay.
Questions (1)
question
- Opening research question of the paper.
Venues (1)
venue
- arXivmentionsPublication venue for the preprint (arXiv:2605.01148).