finding
active
finding:task-accuracy-on-causalgym-increases-consistently-with-model-scale-from-0-62-14m-to-0-89-6-9bTask accuracy on CausalGym increases consistently with model scale from 0.62 (14M) to 0.89 (6.9B)
Scaling result showing larger pythia models perform better on CausalGym linguistic tasks
Source paper
extracted_from(2024) · Aryaman Arora · Dan Jurafsky · Christopher Potts
Neighborhood — ranked by edge-count
Claims (1)
claim
- Forward-looking assertion in conclusion about the lasting value of causal evaluation
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Fig. 9.
- Quantifies performance cost of fine-tuning and steering; deployment steering has minimal accuracy cost.
- Appendix C.
- DAS consistently finds the most causally-efficacious features across all pythia model sizes in CausalGymfinding0.767Main benchmark result showing DAS superiority over probing, diff-in-means, PCA, k-means, LDA, and random
- 1B model batch-gradient similarity to task reference is 0.08±0.02 at injection steps; 300M is 0.04±0.04finding0.764Fig. 9.
- Trend observed in Experiment 2 results.
- DAS achieves overall odds-ratio of 10.24 on pythia-410m averaged across all CausalGym tasksfinding0.760Numerical result for pythia-410m
- Identified limitation calling for broader task coverage in future work