claim
active
claim:gains-on-unsupervised-proxy-metrics-do-not-reliably-translate-to-better-practical-performanceGains on unsupervised proxy metrics do not reliably translate to better practical performance
Core thesis.
Source paper
extracted_from(2025) · Adam Karvonen · Can Rager · Johnny Lin · Curt Tigges +11
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (3)
finding
- Central result.
- Frontier ranking differs from other metrics.
- Rankings diverge across metrics.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Acknowledged limitation.
- Filler tokens question.
- Key interpretive claim from Case Study II distinguishing probe accuracy from causal relevance
- Proxy argument.
- what metrics can reliably tell us if dictionary learning has successfully extracted high quality features?question0.746Central methodological gap: current metrics (loss, density histograms, manual inspection) are inadequate
- Stated limitation.
- Shows that truth representations are not reducible to text probability representations
- The paper likely under-reports probe ability because trained probes would outperform DoM vectorsclaim0.734Limitation.