method
active
method:monte-carlo-auroc-averagingMonte Carlo AUROC averaging
Averages AUROC over 500 simulations drawing 1,000 precomputed rollouts per α.
Neighborhood — ranked by edge-count
Papers (1)
paper
- Predicting Rare LLM Failures with 30× Fewer Rolloutscitesmentions
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Computational algorithm mentioned as an example of diverse problem-solving strategies.
- Performance metric for binary classification; used to evaluate pathogenicity prediction.
- Predictions formed by averaging over policy-specific beliefs, weighted by policy probabilities.
- Procedure for sampling 64 random nonnegative combinations of cone basis vectors to evaluate the full cone distribution
- The mechanism by which LLMs generate text: drawing a token from the next-token distribution and appending it to context repeatedly
- Technique to estimate the continuous EI formula by sampling, used in neural network EI calculation.
- Second model system studied; used to show why flat autoregressive LLMs struggle with long-range coherence.
- Classifies prompts as complying at least once in 100,000 rollouts vs never, per rollout budget.