hypothesis

active

hypothesis:we-hypothesize-that-sparse-autoencoders-or-similar-methods-will-work-on-frontier-large-language-models-though-significant-computational-challenges-remain

We hypothesize that sparse autoencoders or similar methods will work on frontier large language models, though significant computational challenges remain

Forward-looking prediction about scalability of the method to larger models

Source paper

extracted_from

Towards Safe and Honest AI Agents with Neural Self-Other Overlap

(2024) · Marc Carauleanu · Michael Vaiana · Judd Rosenblatt · Cameron Berg +1

Neighborhood — ranked by edge-count

Findings (1)

finding

A/5 autoencoder (131,072 features) recovers 94.5% of MLP log-likelihood loss reduction
associated_with
Shows that loss recovery increases with autoencoder size

Claims (1)

claim

Sparse autoencoders extract features that are significantly more monosemantic than neurons, as shown by four independent lines of evidence
extends
Central claim of the paper, supported by detailed feature analysis, human evaluation, automated interpretability of activations, and automated interpretability of logit weights

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Sparse autoencoders produce interpretable features for large models.claim0.878
Central claim of the paper: the method scales to state-of-the-art transformers.
Sparse Autoencoders Find Highly Interpretable Features in Language Models (Cunningham et al., 2023)concept0.871
Core methodology paper for SAE-based interpretable feature extraction
Sparse autoencoders are preferable to stronger iterative dictionary learning methods because they cannot recover features the model itself cannot accessclaim0.858
Rationale for using simpler sparse autoencoders rather than NP-hard compressed sensing algorithms
Sparse autoencoders don't provide a comprehensive solution because they decode activations, not parametersclaim0.844
Critique of activation-based interpretability methods.
Training the sparse autoencoder on more data makes features subjectively sharper and more interpretableclaim0.807
Empirical principle discovered during autoencoder training; led to using 8 billion training points
Sparse Autoencoderframework0.797
Interpretability framework used to decompose layer-40 activations into sparse feature sets for studying emotional alignment and persistence
Sparse Autoencoder Featuresconcept0.795
Used in Anthropic welfare assessment to identify performative behavior and hidden emotional struggle co-activating features
Autoregressive language models cannot converge to single stored patterns beyond their context window from local interactions alone.claim0.795