question
active
question:does-esr-reflect-model-scale-architecture-or-training-proceduresDoes ESR reflect model scale, architecture, or training procedures?
Central unresolved question about the mechanism behind ESR's apparent size-dependence
Source paper
extracted_from(2026) · Alex McKenzie · Keenan Pepper · Stijn Servaes · Martin Leitgab +5
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- We cannot isolate whether ESR reflects scale, architecture, or training procedures in Llama-3.3-70BgatesEpistemic limitation claim acknowledging confounds in the cross-model comparison
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- The observed pattern that ESR appears predominantly in the largest model tested, suggesting scale-dependence
- Inverse SCR scaling persists across intervention sizes for non-hierarchical architecturesfinding0.755Rules out fraction-modified explanation.
- We hypothesize ESR may emerge from RLHF training rather than existing in pretrained representationshypothesis0.751Open question about the developmental origin of ESR mechanisms
- Edge weights first-order approximation.
- Are there model structures that account for memorization and generalization across large swaths of training data?question0.742Aggregate analysis motivation.
- Core goal statement.
- Central interpretive claim from statistical analysis