finding
active
finding:some-activation-capping-settings-slightly-improve-performance-on-ifeval-mmlu-pro-or-gsm8k-for-both-qwen-and-llamaSome activation capping settings slightly improve performance on IFEval, MMLU Pro, or GSM8k for both Qwen and Llama
Unexpected positive finding suggesting capping may sometimes help capabilities
Source paper
extracted_from(2026) · Christina Lu · Jack Gallagher · Jonathan Michala · Kyle Fish +1
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Main quantitative result demonstrating effectiveness of activation capping
- Only model showing marginal benefit from increased reflection, at substantial token cost
- Optimal activation capping layers for Llama 3.3 70B are layers 56-71 (out of 80) at 25th percentile capfinding0.779Specific implementation finding for Llama capping parameters
- Causal evidence that massive activations are required for stages of inference to emerge in looped models
- Activation alignment algorithm succeeded on 99.2% of Llama/Qwen and 99.1% of GPT-OSS/Deepseek sequences.finding0.755Alignment failure analysis.
- Figure 5.
- Optimal activation capping layers for Qwen 3 32B are layers 46-53 (out of 64) at 25th percentile capfinding0.746Specific implementation finding for Qwen capping parameters
- Model-specific baseline personality difference revealed by revealed preferences experiment