finding
active
finding:first-sparse-autoencoders-trained-on-the-671b-parameter-deepseek-r1-open-sourced-as-two-saesFirst sparse autoencoders trained on the 671B-parameter DeepSeek R1, open-sourced as two SAEs
Headline result: first public interpreters on a true reasoning model at this scale.
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Central interpretive assertion of the post.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Central claim of the paper: the method scales to state-of-the-art transformers.
- Interpretability method criticized in this paper for shattering manifolds into atomic pieces, obscuring overarching semantic structure.
- Sparse Autoencoders Find Highly Interpretable Features in Language Models (Cunningham et al., 2023)concept0.809Core methodology paper for SAE-based interpretable feature extraction
- Critique of activation-based interpretability methods.
- Sparse dictionary learning method used to extract interpretable features from EEG transformer embeddings.
- Interpretability framework used to decompose layer-40 activations into sparse feature sets for studying emotional alignment and persistence
- SAEs trained on 100M+ tokens to compress token layer-40 activations into 64 active features out of 100K+ for interpretability analysis
- Empirical principle discovered during autoencoder training; led to using 8 billion training points