framework
active
framework:turn-averaged-saeTurn-Averaged SAE
Paper's main proposal: SAE trained on the mean activation across a turn.
Neighborhood — ranked by edge-count
Papers (1)
paper
- Turn-Averaged SAEs for Feature Discovery and Long-Context Attributionintroducesmentions
Methods (1)
method
- Reconstruction quality metric.
Concepts (1)
concept
- Turn averaging as low-pass filterimplementsIntuition for why averaging yields high-level features.
Frameworks (4)
framework
- Method using large language models (Claude) to generate and test explanations of features at scale
- Nested SAEusesHybrid SAE jointly training turn-averaged inner and per-token outer partitions.
- Decoupled SAEusesHybrid with separate turn-averaged and residual per-token SAEs.
- Temporal SAEscontradictsContrastive loss approach for smooth features across adjacent tokens; contrasted with turn averaging.
Datasets (2)
dataset
- LMSYS-CHAT-1McitesChat dataset (n=18,777 sampled) used to measure how much persona space PCs explain overall activation variance and to calibrate steering norms
- Qwen 2.5 7B-InstructcitesCross-family model used for replication of wellbeing concept analysis; shows strong introspection (ρ=0.49, R²=0.76)
Findings (1)
finding
- Appendix G.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Table 1 on Qwen-2.5-7B holdout.
- Table 1.
- Table 1.
- Broadly, turn-averaged SAEs make interpretability techniques practical at long context lengths.quote0.776Abstract thesis.
- Attribution graphs with nodes per turn per layer.
- Abstract.
- Turn-averaged SAE trains in ~1.5 hours on 4×H100 with ~1.58M samples versus 266M tokens per-tokenfinding0.742Training efficiency.
- Standard baseline SAE operating on individual token activations.