claim
active
claim:large-virtual-weights-are-not-inherently-misleading-but-not-guaranteed-to-be-helpful-or-effectiveLarge virtual weights are not inherently misleading but not guaranteed to be helpful or effective
Caution for practitioners.
Source paper
extracted_fromNicholas L. Turner · Jeffrey Wu · Joshua Batson
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (3)
finding
- Naive weight magnitude works here.
- Fisher effectiveness is roughly lognormal across 10 orders of magnitude; median is 10,000× below maxsupportsVersus VW approx normal, median one third of max.
- Helpful and harmful weights are scattered across the entire range of virtual weight magnitudessupportsIN Tokens→Logits shows weak VW-helpfulness relationship.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Motivating example.
- Three regimes by effectiveness.
- Contrast with answer selection.
- Implicit weights directly connecting any pair of layers computed by multiplying output weights of one layer with input weights of another through the residual stream
- Used to project features through QK weights and find aligned features.
- Load-bearing claim about the tractability of circuit analysis; central thesis of Claim 2
- With raw data, wider token strides give more accurate approximations at same token budget.finding0.721Cost tradeoff.
- Even restricted to most effective and helpful.