concept
active
concept:pretraining-exposure-densityPretraining exposure density
Expected prevalence of patterns (e.g., base-10 arithmetic) in pretraining corpora, influencing ρd and dr.
Neighborhood — ranked by edge-count
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Interpretation that pattern density from pretraining determines few-shot requirements
- Architectural modification subtracting a learned bias from autoencoder inputs before encoding; initialized to geometric median of dataset; improves autoencoder performance
- Hypothesis: Shot midpoint ordering k50(B10) < k50(B8) ≈ k50(B9) follows pretraining exposure densityhypothesis0.719E2 prediction that bases with higher pretraining exposure require fewer shots to cross threshold
- Approximate posterior probability distribution embodied in organism's internal states; organism's best guess about causes of sensations
- Pretraining stores latent patterns that coherent anchors can bind (or misbind) to targets.quote0.709Load-bearing quote capturing the core metaphor
- Finding that base models have high false positives and no net positive performance.
- Developmental analogy used to explain sample efficiency under high ρd conditions
- The plasticity or willingness of cells to move; raised by stress sharing, analogous to annealing temperature.