paper
active
paper:aranguri-predicting-rare-b9e07b

Predicting Rare LLM Failures with 30× Fewer Rollouts

TL;DR

Rates of rare harmful behavior in Qwen 3 4B can be estimated with up to 30× fewer rollouts than naive sampling by borrowing statistical strength from a less-safe sibling model. Logit Path Extrapolation interpolates between Qwen 3 4B and Qwen 3 4B Abliterated (refusal direction removed) in logit space, with M(α) sampled autoregressively from the mixed logits. It measures compliance at values of α where compliance is common, then fits a line to log₁₀(compliance) and extrapolates to α=1. On the 200 HarmBench standard prompts, the task is to separate prompts that never comply in 100 rollouts from those that comply at least once in 100,000 rollouts. Scored by AUROC, the method matches naive sampling with 20× fewer rollouts at 100 rollouts per α. A cheaper search for the starting α (10 rollouts per α) raises this to 28×, and to about 30× at the best budget. It beats importance sampling, the prior state of the art, which gave 10-20× gains in related work. The load-bearing empirical regularity is that log compliance is approximately linear in α below 50% compliance. The paper argues that rare failures can be predicted from a model where they are common, and that the same procedure applies to any pass/fail check on model outputs, including whole red-teaming pipelines. That could make pre-deployment estimation of one-in-a-million failures tractable.

What to take away

  1. 1. Logit Path Extrapolation matches the AUROC of naive sampling with up to 30× fewer rollouts when classifying HarmBench prompts that comply at least once in 100,000 rollouts of Qwen 3 4B versus prompts that never do.
  2. 2. With 100 rollouts per α for both search and fit, the method reaches naive sampling's AUROC using 20× fewer total rollouts.
  3. 3. Splitting extrapolation into a 10-rollouts-per-α search for the starting α and a 100-rollout fit raises the saving to 28×, with up to 30× when the fit rollouts are varied.
  4. 4. The method outperforms importance sampling, using Qwen 3 4B Abliterated as the proposal distribution, whereas related work reports only 10-20× gains for importance sampling after optimizing the base distribution.
  5. 5. The interpolated logits define an effective model M(α) that is sampled autoregressively, re-computing both models' logits after each sampled token, which extends Logit Diff Amplification from extrapolating beyond the model to interpolating toward it.
  6. 6. The core empirical regularity is that log P(M(α) complies) is approximately linear in α when compliance is not close to 1, and the fit uses only α values with compliance below 50%.
  7. 7. Replicable procedure: scan α from 0.95 toward 0 in 0.05 steps with 10 rollouts each, stop at ≥50% compliance, then draw n_fit (13 to 100) rollouts per α back toward 1 and stop at the first α with zero compliant rollouts.
  8. 8. If fewer than two α values show nonzero compliance, the method defaults to a 10⁻⁶ prediction, which happens for fewer than 1 in 100 prompts on average.
  9. 9. Evaluation averages AUROC over 500 Monte Carlo simulations on subsets of 1,000 precomputed rollouts per α, with naive sampling simulated as Binomial(k, p) from 100,000 stored rollouts and compliance judged by GPT-5.4-mini using the StrongREJECT refusal rubric at temperature 1.
  10. 10. Open question: whether the log-linear relation holds beyond refusal abliteration, such as for other behaviors, multi-turn red-teaming pipelines or larger models, since the authors flag it as empirical and the method needs a second model where the behavior is more common.

Peer brief — for seminar discussion

The work tackles pre-deployment estimation of rare harmful behavior: a failure at one in a million rollouts will not show up in testing but will appear after release. It introduces Logit Path Extrapolation. Take Qwen 3 4B and Qwen 3 4B Abliterated, where the refusal direction is removed. Sample autoregressively from logits interpolated by α between them. Measure compliance at α values where compliance is common, fit a line to log₁₀ compliance using points under 50%, and extrapolate to α=1. The test set is the 200 HarmBench standard prompts. The task is to separate prompts that never comply in 100 rollouts from those that comply at least once in 100,000 rollouts. Performance is AUROC against rollout budget, with compliance judged by GPT-5.4-mini using the StrongREJECT rubric, averaged over 500 Monte Carlo draws. The headline is a 20× saving over naive sampling at 100 rollouts per α, rising to 28× with a cheap 10-rollout search for the starting α, and about 30× at the best budget. Importance sampling with the abliterated model as proposal, the closest alternative, is clearly beaten. Related work reports 10-20× gains for it, and the self-normalized variant did worse here. Underlying this is a hypothesis that log compliance is roughly linear in α. The linearity is only observed, not derived. The implication is that a less-safe sibling, which labs usually have, is a usable instrument for forecasting tail risk. Any pass/fail check, including whole red-teaming pipelines, could be run on M(α) and extrapolated. This differs from Jones et al.'s forecasting of the worst risk across a dataset of prompts, and from Logit Diff Amplification, which surfaces rare behaviors rather than estimating their probability. A critical reader should push on scope and validity. The evidence is a single 4B model, one abliteration-based variant, and one benchmark. The ground truth is itself only 100,000 rollouts, so true rates below about 10⁻⁵ are unresolved, and the evaluation is a binary ranking (AUROC) rather than calibration of the estimated probabilities. A method could rank prompts well while being badly miscalibrated in absolute rate, and absolute rate is what a deployment decision needs. The 10⁻⁶ fallback and the stopping heuristics are further tuning choices. Abliteration may also produce a path whose geometry is unusually smooth, so the log-linear fit may not transfer to behaviors with no clean linear direction. A natural alternative would be importance sampling with an optimized proposal or a learned tilted distribution, which the paper does not tune as far.

Methods (12)

Datasets (2)

Related work— refs + corpus + external arXiv

Cited / in-corpus / arXiv badges show which signals surfaced each row. Multi-source rows weighted higher.

Similar preprints — Semantic Scholar