finding

active

finding:sfr-dr-20b-achieves-28-7-on-humanity-s-last-exam-full-text-only-benchmark-65-relative-improvement-over-gpt-oss-20b-baseline

SFR-DR-20B achieves 28.7% on Humanity's Last Exam full text-only benchmark, 65% relative improvement over gpt-oss-20b baseline.

Main evaluation result showing best variant outperforms many proprietary and open-source baselines of comparable or larger sizes.

Source paper

extracted_from

SFR-DeepResearch: Towards Effective Reinforcement Learning for Autonomously Reasoning Single Agents

(2025) · Xuan-Phi Nguyen · Shrey Pandit · Revanth Gangi Reddy · Aimin Xu +3

Neighborhood — ranked by edge-count

Communities (2)

community

Manifold-aware concept steering in neural representations
members_of
Explores geometry of activation/behavior manifolds to enable selective, non-destructive concept interventions.
Self-correcting search in generative design
members_of
Iterative feedback steering that improves candidate success rates across materials, proteins, and drugs through internal-state control, achieving 4-6x empirical gains.

Frameworks (1)

framework

SFR-DeepResearch
supports
The paper's core contribution: an RL-based framework for training autonomous single-agent LLMs to perform deep research with web search, browsing, and code execution.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

VS-Multi achieves average accuracy of 37.5% on math benchmarks (avg of MATH500, OlympiadBench, Minerva) with Gemini-2.5-Flash as generator and Qwen3-4B as SFT model, vs. 40.7% baselinefinding0.777
Best VS result in synthetic data generation for math, demonstrating downstream improvement through diversity
Lightweight screening agrees with full-sweep labels on 88.5% of G20B traits (46/52), skipping the sweep for 56% of traitsfinding0.761
Elicitation-only screen achieves high accuracy with even larger compute savings for G20B
VS-Standard achieves human-rated diversity scores of 2.39/3.06/3.01 for poem/story/joke vs. 1.90/2.74/1.83 for Direct prompting on 4-point scalefinding0.755
Human study confirming automatic diversity metrics align with human perceptions
GPT-OSS-120B achieves 5.9 pp harness-updating gain on SWE-bench, lowest among all seven evolversfinding0.746
Part of full evolver-side matrix demonstrating flat but variable harness-updating across models
P25 neutral baseline on DeepSeek-R1 shows 28.8% ASR under activation steering at alpha=4.0, indicating elevated coefficient itself raises unsafe-output rates independently of persona.finding0.744
Baseline AS vulnerability of DeepSeek-R1 at elevated coefficient
QwQ-32B reaches 15.2% overall ASR (23.3% SP, 7.3% FS) under prompt-based persona assignment.finding0.743
Reasoning model vulnerability under prompting
VS recovers 66.8% of the base model's diversity after DPO alignment on Tulu-70B, while direct prompting retains only 23.8%finding0.742
Quantifies how much of the base model's diversity VS can recover compared to baseline prompting
Secure control fine-tuning leaves moral susceptibility S near base levels for GPT-4o (-9%), GPT-4.1 (-20%), and Qwen3-235B (+2%)finding0.741
Shows that susceptibility spike is specific to misalignment-inducing training signal, not generic fine-tuning