concept
active
concept:multitask-scalingMultitask Scaling
The pressure on models trained on more tasks to find representations that generalize across all tasks, reducing the solution space
Neighborhood — ranked by edge-count
Concepts (1)
concept
- Representational ConvergencesupportsThe central empirical phenomenon: different neural networks trained on different data/objectives develop increasingly similar representations
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Argues that there are fewer representations competent for N tasks than M<N tasks, so more general models have a smaller solution space
- Used in the color cooccurrence experiment to embed colors into 3D space preserving dissimilarity matrix distances
- The nesting of overlapping, competing, and cooperating agents at different levels of organization, each with its own goals and cognitive boundary.
- Systems comprising nested hierarchies (cells→organisms→societies); central to understanding collective intelligence
- Hierarchical goal-seeking at multiple levels of organization.
- Approach using extra compute at test time to double-check answers and improve reliability.
- Techniques that leverage AI to help humans more efficiently supervise AI.
- Learning paradigm that jointly learns multiple related tasks using a single model