finding
active
finding:decoupled-sae-has-per-token-nmse-0-212-worse-than-single-model-architecturesDecoupled SAE has per-token nMSE 0.212, worse than single-model architectures
Table 1.
Source paper
extracted_from(2026) · Kevin Der · Harish Kamath · Thompson, Ben
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Per-token SAE input is invariant to uniform shifts from ablating turn-averaged features.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Table 1.
- Table 1.
- Table 1.
- Table 1.
- Hybrid with separate turn-averaged and residual per-token SAEs.
- The objective function combining L2 reconstruction error and L1 penalty scaled by decoder norm, used to train the SAE.
- Assistant-only SAEs reconstruct user turns 1.5–3× worse; both-roles training cuts user-turn nMSE 53–63%finding0.767Appendix G.
- Table 1.