finding
active
finding:trained-models-slightly-exceed-opus-reference-on-unfiltered-bank-0-015-qwen3-8b-0-033-qwen3-5-397bTrained models slightly exceed Opus reference on unfiltered bank (+0.015 Qwen3-8B, +0.033 Qwen3.5-397B)
Gains concentrated in low-concreteness claims.
Source paper
extracted_from(2026) · Adam Karvonen · Euan Ong · Subhash Kantamneni · Samuel Marks
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- From unfiltered results.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- In- and out-of-distribution generalization.
- Core evidence that model withholds pro-animal-welfare responses during training
- Opus 4.6 achieves HFR of 0.757 while Qwen3-32B achieves HFR of only 0.142 on SkillsBenchfinding0.770Quantifies harness adherence failure gap between strong and weak tier models
- Generalization to hint setting.
- Filters do not drive the null.
- Demonstrates severity of training-deployment gap after RL
- Strong-tier model maintains harness adherence over long-horizon trajectories
- Core finding demonstrating non-monotonic relationship between base capability and harness-benefit