finding
active
finding:turn-averaged-sae-achieves-turn-averaged-nmse-0-097-on-its-own-signalTurn-averaged SAE achieves turn-averaged nMSE 0.097 on its own signal
Table 1 on Qwen-2.5-7B holdout.
Source paper
extracted_from(2026) · Kevin Der · Harish Kamath · Thompson, Ben
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Central claim.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Table 1.
- Table 1.
- Table 1.
- Paper's main proposal: SAE trained on the mean activation across a turn.
- Table 1.
- Assistant-only SAEs reconstruct user turns 1.5–3× worse; both-roles training cuts user-turn nMSE 53–63%finding0.792Appendix G.
- Broadly, turn-averaged SAEs make interpretability techniques practical at long context lengths.quote0.761Abstract thesis.
- Turn-averaged SAE trains in ~1.5 hours on 4×H100 with ~1.58M samples versus 266M tokens per-tokenfinding0.759Training efficiency.