finding
active
finding:datapoint-feature-formation-is-controlled-by-the-product-of-repetition-fraction-and-sparsity-regularization-strengthDatapoint feature formation is controlled by the product of repetition fraction and sparsity regularization strength
Hyperparameter scan result.
Neighborhood — ranked by edge-count
Papers (1)
paper
Hypotheses (1)
hypothesis
- Datapoint features form only if there is a repeated datapoint in training and sparsity regularization is appliedassociated_withsupportsTwo necessary conditions.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Motivates Jacobian connection.
- Memorization by transcoder not in model.
- Demonstrates unfaithfulness in a simple setup.
- Playful and poetic amplification support.
- Major prediction checked against InceptionV1.
- Training distribution where special point p=[1,1,1,0,...] appears with fraction repeat_frac.
- Property that features activate on only a small fraction of inputs; enables compressed sensing and is what allows superposition to work
- Enables pre-finetuning data screening; Figure 8 shows strong dataset-level correlations across all three traits