finding
active
finding:many-metrics-reach-most-performance-by-50m-tokens-of-500m-on-16k-topk-and-relu-saesMany metrics reach most performance by 50M tokens of 500M on 16k TopK and ReLU SAEs
Training dynamics.
Source paper
extracted_from(2025) · Adam Karvonen · Can Rager · Johnny Lin · Curt Tigges +11
Neighborhood — ranked by edge-count
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Central result.
- Exception to ReLU underperformance.
- Table 1.
- Turn-averaged features have highest embedding similarity 0.663 at k=3; per-token max lowest at 0.617finding0.776Consistent across k=1,2,3,5.
- Basic SAE performance metrics.
- Design justification.
- §5.5.
- Dropping 50% of monitor calls on SWE-bench with GLM lowers precision 0.656→0.588 while Kimi drops ~0.01finding0.756Cost of replacement.