Optimal number of features scales faster than optimal number of training steps with compute budget.

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Optimal learning rate decreases as a power law with compute budget.finding0.813
  Hyperparameter trend observed.
• The likelihood of a dedicated feature for a concept (element, city, animal, food) follows a sigmoid in log-frequency of the concept in training data, with threshold frequency inversely proportional to number of alive features.finding0.774
  Quantitative relationship between concept frequency and feature presence.
• There are fewer representations competent for N tasks than M<N tasks, so training more general models should yield fewer possible solutionshypothesis0.774
  Selective pressure toward convergence via task generality
• what is the 'correct number of features' for dictionary learning, and is this question well-posed?question0.772
  Open question about whether there is a true discrete feature count or a continuous splitting process
• SAE training loss decreases as a power law with compute budget when using compute-optimal hyperparameters.finding0.764
  From scaling laws sweep.
• There appears to be a systematic relationship between the frequency of concepts and the dictionary size needed to resolve features for them.claim0.762
  Feature presence depends on concept frequency in training data, with a threshold scaling inversely with alive features.
• The multiscale competency architecture (MCA) speeds evolutionary search by providing generalization, reliability, tractable search space, cryptic variation, and functional intermediates.claim0.759
  Main functional claim about MCA.
• Features are connected by weights forming circuits, and these circuits can be rigorously studied and understood as meaningful algorithms.claim0.757
  Second of three speculative claims asserting that subgraphs of neural networks are tractable and meaningful objects of study