question
active
question:do-we-assume-that-the-model-knows-something-we-don-t-or-are-these-interference-weights-contributing-noiseDo we assume that the model knows something we don't, or are these interference weights contributing noise?
Motivating question.
Source paper
extracted_fromNicholas L. Turner · Jeffrey Wu · Joshua Batson
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (2)
finding
- Largest IN Tokens→Logits weight votes 'utions', which never follows IN in training; Fisher effectiveness ~3 orders below the top weights.
- No single path ranks E top.
Claims (1)
claim
- Interpretation of motivating example.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Concluding statement.
- The model tends to reflect more when the question is difficult, and accuracy is generally lower for harder questionshypothesis0.794Hypothesis explaining negative correlation between reflection rate and accuracy without implying reflection is harmful
- Key jailbreak insight.
- Instead of waiting for a model to verbalize eval awareness, we measure how likely it is to do so.quote0.781Core idea of the paper.
- Implication of PRH for AI fairness and bias
- Self-explanation vs circuit.
- Intro question.
- Abstract claim.