finding
active
finding:evaluating-one-sae-across-all-benchmarks-takes-roughly-65-110-minutes-on-an-rtx-3090-with-107-152-minutes-setupEvaluating one SAE across all benchmarks takes roughly 65-110 minutes on an RTX 3090 with 107-152 minutes setup
Computational cost.
Source paper
extracted_from(2025) · Adam Karvonen · Can Rager · Johnny Lin · Curt Tigges +11
Neighborhood — ranked by edge-count
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Turn-averaged SAE trains in ~1.5 hours on 4×H100 with ~1.58M samples versus 266M tokens per-tokenfinding0.778Training efficiency.
- DAS runtime is invariant with number of testing hypotheses, unlike brute-force search.
- Fig 2 right.
- Quantifies SAE reconstruction damage.
- SAE Feature #92372 fires 666,235 times in corpus, associated with urgency vs. receptive calm dimensionfinding0.703Example of a highly active SAE feature modulating urgency versus acceptance as an emotional dimension
- Exception to ReLU underperformance.
- Documents the prohibitive cost of exhaustive auditing without screening