Nucleus Sampling

Decoding strategy used throughout experiments; p=0.9 selected to increase response diversity

Neighborhood — ranked by edge-count

paper

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Nucleus sampling generates more semantically diverse utterances than beam searchclaim0.764
New semantic diversity dimension added to prior finding that nucleus sampling is more lexically diverse
Monte Carlo Cone Samplingmethod0.732
Procedure for sampling 64 random nonnegative combinations of cone basis vectors to evaluate the full cone distribution
Autoregressive Samplingmethod0.730
The mechanism by which LLMs generate text: drawing a token from the next-token distribution and appending it to context repeatedly
Rejection samplingmethod0.726
A technique to filter model outputs; Redwood Research's project mentioned.
Thompson Samplingmethod0.718
A Bayesian exploration strategy that samples from the posterior distribution over model parameters to decide actions.
Activation Interval Samplingmethod0.711
Dividing feature activation spectrum into 11 evenly-spaced intervals and sampling uniformly to evaluate monosemanticity across activation levels
Probabilistic Bisection Algorithmmethod0.690
Algorithm used to calibrate per-latent threshold boost values for consistent first-attempt difficulty
Persona Sampling Hypothesisconcept0.686
Hypothesis that LLM is sampling from distribution of personas; a consistent fraction of which align-fake, explaining correlation between AF reasoning and compliance gap