Model-Diffing with Sparse Autoencoders

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Model Diffingconcept0.839
  Approach to identify interpretable differences between a base LLM and its fine-tuned version using SAE latents
• Sparse Autoencoderframework0.833
  Interpretability framework used to decompose layer-40 activations into sparse feature sets for studying emotional alignment and persistence
• Sparse Autoencoders (SAE)method0.820
  Interpretability method criticized in this paper for shattering manifolds into atomic pieces, obscuring overarching semantic structure.
• Sparse Autoencoder Featuresconcept0.817
  Used in Anthropic welfare assessment to identify performative behavior and hidden emotional struggle co-activating features
• Sparse autoencoders produce interpretable features for large models.claim0.808
  Central claim of the paper: the method scales to state-of-the-art transformers.
• Sparse Autoencoders Find Highly Interpretable Features in Language Models (Cunningham et al., 2023)concept0.806
  Core methodology paper for SAE-based interpretable feature extraction
• Sparse Autoencoder for Dictionary Learningframework0.797
  Primary method introduced: trains a one-hidden-layer MLP with L1 sparsity penalty to decompose model activations into overcomplete feature dictionaries
• Sparse autoencoders are preferable to stronger iterative dictionary learning methods because they cannot recover features the model itself cannot accessclaim0.793
  Rationale for using simpler sparse autoencoders rather than NP-hard compressed sensing algorithms