framework
active
framework:matryoshka-saesMatryoshka SAEs
Nested dictionary approach that inspires the nested loss.
Neighborhood — ranked by edge-count
Papers (2)
paper
- Turn-Averaged SAEs for Feature Discovery and Long-Context Attributionintroducesmentions
Frameworks (2)
framework
- Matryoshka BatchTopK SAErelated_toHierarchical architecture that excels on disentanglement and scales positively.
- Nested SAEextendsHybrid SAE jointly training turn-averaged inner and per-token outer partitions.
Hypotheses (1)
hypothesis
- Proposed remedy.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Architecture compared.
- Evaluated architecture (Gao et al.).
- Central result.
- Evaluated architecture.
- Sparse dictionary learning method used to extract interpretable features from EEG transformer embeddings.
- Contrastive loss approach for smooth features across adjacent tokens; contrasted with turn averaging.
- SAEs trained on pretrained Gemma-2 models used for steering in Gemma family experiments
- Matryoshka's hierarchical structure explains its positive scaling on disentanglement metricsclaim0.663Mechanistic interpretation.