hypothesis
active
hypothesis:frequently-repeated-chat-features-e-g-knowledge-cutoff-are-memorized-by-the-sae-and-base-model-causing-isolationFrequently repeated chat features (e.g. knowledge cutoff) are memorized by the SAE and base model, causing isolation
Second explanation for isolated features.
Neighborhood — ranked by edge-count
Papers (1)
paper
- Mapping the Latent Space of Llama 3.3 70B - Goodfirecitesmentions
Findings (1)
finding
- Observation on the Llama 3.3 70B SAE feature map.
Concepts (1)
concept
- Possible explanation for isolated repetitive chat features.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Feature cluster examples from the UMAP.
- Chat-trained SAEs find behaviourally relevant features because many causal features occur on chat-specific special tokenshypothesis0.775Explanation for LMSYS superiority.
- Similar number and nature of exclusive features across models results from oversampling of Human/Assistant chat transcriptshypothesis0.771Motivated by Bricken et al; tested and not supported.
- Surprising finding that the two evaluation methods diverge in their relationship with persistence
- Core critique of sparse autoencoders: they break the geometric structure of representations, making it harder to see the big picture.
- E.g. 'Here is a draft', apology, and 'explain like I'm in sixth grade' features.
- SAE latents depend on training data.