claim
active
claim:turn-averaged-saes-make-interpretability-practical-at-long-context-by-decoupling-feature-count-from-token-countTurn-averaged SAEs make interpretability practical at long context by decoupling feature count from token count
Abstract.
Source paper
extracted_from(2026) · Kevin Der · Harish Kamath · Thompson, Ben
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (2)
finding
- Pruning reduces 14-turn graph from 7,239 nodes and 10.4M edges to ~1,300 nodes and ~110K edgessupportsCircuit-tracer pruning.
- Turn-averaged SAE trains in ~1.5 hours on 4×H100 with ~1.58M samples versus 266M tokens per-tokensupportsTraining efficiency.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Broadly, turn-averaged SAEs make interpretability techniques practical at long context lengths.quote0.912Abstract thesis.
- Appendix I.
- Example A.
- Prediction from coverage results.
- Example C.
- Simple interpretability methods scale to 2.8T-parameter models and long-context agentic rolloutsclaim0.778Scaling claim.
- §5.5.
- Surprising finding that the two evaluation methods diverge in their relationship with persistence