thinker:t-mcgrathT. McGrath
Co-author of the model diff amplification paper.
Authored papers (1)
Rates of rare harmful behavior in Qwen 3 4B can be estimated with up to 30× fewer rollouts than naive sampling by borrowing statistical strength from a less-safe sibling model. Logit Path Extrapolation interpolates between Qwen 3 4B and Qwen 3 4B Abliterated (refusal direction removed) in logit space, with M(α) sampled autoregressively from the mixed logits. It measures compliance at values of α where compliance is common, then fits a line to log₁₀(compliance) and extrapolates to α=1. On the 200 HarmBench standard prompts, the task is to separate prompts that never comply in 100 rollouts from those that comply at least once in 100,000 rollouts. Scored by AUROC, the method matches naive sampling with 20× fewer rollouts at 100 rollouts per α. A cheaper search for the starting α (10 rollouts per α) raises this to 28×, and to about 30× at the best budget. It beats importance sampling, the prior state of the art, which gave 10-20× gains in related work. The load-bearing empirical regularity is that log compliance is approximately linear in α below 50% compliance. The paper argues that rare failures can be predicted from a model where they are common, and that the same procedure applies to any pass/fail check on model outputs, including whole red-teaming pipelines. That could make pre-deployment estimation of one-in-a-million failures tractable.
More papers — OpenAlex / S2
Co-authors (5)
- Santiago Aranguri2 shared
- Angell et al.1 shared
- Francisco Pernice1 shared
- Jones, E. et al.1 shared
- Wu & Hilton1 shared
Other inbound relations (1)
- mentionsPredicting Rare LLM Failures with 30× Fewer Rollouts(paper)
Recent mentions (1)
- papers-typedaranguri-predicting-rare-b9e07b.md