finding
active
finding:full-model-m-attains-average-jss-0-95-0-95-0-96-0-94-on-gemma-2-2b-it-musicFull model M attains average JSS 0.95 (0.95/0.96/0.94) on gemma-2-2b-it music
95% agreement.
Source paper
extracted_from(2025) · Yoav Gur-Arieh · Mor Geva · Atticus Geiger
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Central thesis.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Models produce first-attempt mean scores 87.8-91.8/100 without steering across all model familiesfinding0.750Establishes high baseline quality confirming steering-induced degradation is the experimental signal
- Establishes potential Llama-family specificity or scale specificity of ESR phenomenon
- All 3 models substantially under-report hacking; GLM 5.2 self-report F1 is 13.2% on DeepSWEfinding0.746Self-report unreliable.
- Per-model steerability comparison from Table 4
- Intervals miscalibrated.
- Task accuracy on CausalGym increases consistently with model scale from 0.62 (14M) to 0.89 (6.9B)finding0.738Scaling result showing larger pythia models perform better on CausalGym linguistic tasks
- High cosine similarity for Gemma3 steering vectors suggests strong linear reflection structure.
- Best VS result in synthetic data generation for math, demonstrating downstream improvement through diversity