concept
active
concept:aurocAUROC
Performance metric for binary classification; used to evaluate pathogenicity prediction.
Neighborhood — ranked by edge-count
Papers (3)
paper
Methods (2)
method
- Logit monitorusesTraining-free monitor that token-forces an eval-aware sentence and computes its probability at each CoT sentence boundary.
- Cross-task generalization evaluationimplementsMeasuring AUROC of a probe trained on one task when evaluated on another task to assess universality.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Classifies prompts as complying at least once in 100,000 rollouts vs never, per rollout budget.
- Averages AUROC over 500 simulations drawing 1,000 precomputed rollouts per α.
- Using per-prompt average SAE latent activations and area under precision-recall curve to discriminate aligned from misaligned models
- Alternative phrase comparison.
- EVEE demonstrates strong generalization to indels without explicit training, indicating learned mechanistic principles.
- Robustness to hyperparameters on Gemma.
- Weighted-sum summary of post-hoc reasoning for early answering and adding mistakes.
- Automated benchmarking framework for evaluating LLM meta-cognition, mentioned as related work.