hypothesis
active
hypothesis:base-models-assign-higher-likelihood-to-typical-set-representative-sequences-than-to-degenerate-sequences-under-vs-promptsBase models assign higher likelihood to typical-set (representative) sequences than to degenerate sequences under VS prompts
Assumption D.6 formalized in the theoretical framework; empirically validated with coin-flip typicality rating experiments
Source paper
extracted_from(2025) · Jiayi Zhang · Simon C.H. Yu · Derek Chong · Anthony Sicilia +3
Neighborhood — ranked by edge-count
Papers (1)
paper
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Shows typicality bias is preserved through instruction tuning and RLHF, not introduced by alignment
- Supported by empirical comparison showing VS achieves KL divergence of 0.12 from pretraining distribution vs. 14.89 for direct prompting
- Observed by Anima Labs in untrained base models; not present in training data, implying computational origin of self-reported parallel processing.
- Claim that capability emerges from architecture, not data, and that later models lose the surprise.
- Observation about asymmetry in base model capabilities.
- MLP and attention analysis.
- Assumption D.3 formalized in the theoretical framework; empirically validated with coin-flip sequence experiments
- A better basis (e.g., factoring tokens into languages or parts of speech) may yield a sparse interpretable modelhypothesis0.764Same model can be dense in one basis and sparse in another.