method
active
method:strict-loose-accuracy-and-levenshtein-metricsStrict/Loose accuracy and Levenshtein metrics
Evaluation metrics for exact and near-verbatim memorization.
Neighborhood — ranked by edge-count
Papers (1)
paper
- From Memorization to Reasoning in the Spectrum of Loss Curvatureintroducesmentions
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Supervised metrics are limited by ground-truth availability and cover only a fraction of latentsclaim0.751Limitation.
- Metric measuring the mean MSE between self and other-referencing activations across all hidden MLP/attention layers
- Criterion requiring that model's description of internal state be accurate, distinguishing genuine introspection from confabulation.
- Proportion of characters (out of 26) for which all five Big Five dimensions are predicted correctly
- Theoretical limitation identified by the authors distinguishing reflection from stylistic tasks.
- what metrics can reliably tell us if dictionary learning has successfully extracted high quality features?question0.715Central methodological gap: current metrics (loss, density histograms, manual inspection) are inadequate
- Evaluation metric: proportion of samples with predicted answer exactly matching ground-truth, with flexible number extraction.
- With raw data, wider spacing has little effect on accuracy except at high S and near forking points.finding0.711Raw data behavior.