concept
active
concept:internet-scale-training-corpusInternet-scale Training Corpus
The large corpus of human-generated text on which LLMs are trained, which provisions character archetypes and narrative structures
Neighborhood — ranked by edge-count
Concepts (1)
concept
- Large Language Models (LLMs)associated_withTransformer-based models like GPT-4, LaMDA, PaLM; assessed for GWT indicators.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- The shape of the pretraining corpus is a direct lever on which traits a base model can expresshypothesis0.726Forward-looking hypothesis about pretraining data as mechanism for persona formation
- Data generation strategy grounding personality instances in realistic behavioral scenarios rather than abstract self-reports
- Candidate approach to solving superposition.
- Loss gains a smaller model recovers with infinite data.
- Are there model structures that account for memorization and generalization across large swaths of training data?question0.717Aggregate analysis motivation.
- Core goal statement.
- Scaling laws for dictionary learning are unknown and needed to assess feasibility on frontier models
- AI training method inspired by behaviorism, used for autonomous cars and drones; cited as bioinspired success