claim
active
claim:further-sharpening-of-the-effectiveness-metric-is-not-high-leverage-compared-to-finding-better-basesFurther sharpening of the effectiveness metric is not high-leverage compared to finding better bases
Helpfulness is direct measure and Fisher already prunes similarly.
Source paper
extracted_fromNicholas L. Turner · Jeffrey Wu · Joshua Batson
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (3)
finding
- No clean effective/ineffective split.
- ~27,000 point comparison.
- Effectiveness doesn't fully clean.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- With raw data, wider spacing has little effect on accuracy except at high S and near forking points.finding0.766Raw data behavior.
- Implication of PRH for 'scale is all you need' argument
- Acknowledged limitation.
- Author's interpretation of the negative correlation between reflection rate and accuracy observed in Fig. 5
- Practical implication.
- If the relationship is present in activations, better tools (e.g. trained on these datasets) could surface ithypothesis0.743Alternative explanation.
- Better bases may give sparsity.
- Wider stride improves raw accuracy at fixed budget because averaging over a window reduces variance.claim0.741Explanation of raw behavior.