quote
active
quote:our-probe-based-metric-is-quite-noisyOur probe based metric is quite noisy
Gao et al. cited on sparse probing noise.
Source paper
extracted_from(2025) · Adam Karvonen · Can Rager · Johnny Lin · Curt Tigges +11
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (1)
finding
- Probing limited.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Key interpretive claim from Case Study II distinguishing probe accuracy from causal relevance
- Interpretation of Fig. 9.
- Key methodological insight: introspection enables a new probe validation criterion beyond conventional separation metrics
- Authors' claim that their approach is both more effective in reduction and cheaper than prior methods.
- The paper likely under-reports probe ability because trained probes would outperform DoM vectorsclaim0.763Limitation.
- Motivation for causal evaluation over purely behavioural probing accuracy
- Metric introduced to quantify steering selectivity by comparing the area of target and off-target concept probes.
- Key interpretive question the framework helps address through convergent validation logic