finding
active
finding:sfr-dr-20b-achieves-28-7-on-humanity-s-last-exam-full-text-only-benchmark-65-relative-improvement-over-gpt-oss-20b-baselineSFR-DR-20B achieves 28.7% on Humanity's Last Exam full text-only benchmark, 65% relative improvement over gpt-oss-20b baseline.
Main evaluation result showing best variant outperforms many proprietary and open-source baselines of comparable or larger sizes.
Source paper
extracted_from(2025) · Xuan-Phi Nguyen · Shrey Pandit · Revanth Gangi Reddy · Aimin Xu +3
Neighborhood — ranked by edge-count
Communities (2)
community
- Explores geometry of activation/behavior manifolds to enable selective, non-destructive concept interventions.
- Iterative feedback steering that improves candidate success rates across materials, proteins, and drugs through internal-state control, achieving 4-6x empirical gains.
Frameworks (1)
framework
- SFR-DeepResearchsupportsThe paper's core contribution: an RL-based framework for training autonomous single-agent LLMs to perform deep research with web search, browsing, and code execution.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- SFT on correction+retraction data gives 39.4% inline overall reduction and 9.6% not-inline, well below RLFRfinding0.780SFT baseline comparison.
- Best VS result in synthetic data generation for math, demonstrating downstream improvement through diversity
- Benchmark identification rate rises from 0.08 (SFT) to 0.25 (DPO); interventions barely reduce itfinding0.776Table 11.
- RLFR policy matches base on standard benchmarks (e.g., MMLU 67.2 vs 69.2, GSM8K 73.0 vs 72.3)finding0.764No capability degradation.
- Elicitation-only screen achieves high accuracy with even larger compute savings for G20B
- Human study confirming automatic diversity metrics align with human perceptions
- GPT-OSS-120B achieves 5.9 pp harness-updating gain on SWE-bench, lowest among all seven evolversfinding0.746Part of full evolver-side matrix demonstrating flat but variable harness-updating across models
- Baseline AS vulnerability of DeepSeek-R1 at elevated coefficient