paper
active
paper:mcgrath-mapping-latent-95c0a3

Mapping the Latent Space of Llama 3.3 70B - Goodfire

TL;DR

Sparse autoencoders trained on an intermediate layer of Llama 3.3 70B, using LMSys-Chat-1M data, yield a browsable, steerable feature space, and the clearest results are about where it breaks: steering works but silently degrades facts, and LLM-judged scores fail to detect this. Goodfire releases the SAEs through an API and maps the features with a DataMapPlot UMAP visualization, which reveals clusters for biomedical knowledge, physics, programming, cognitive biases, name abstractions, and phonetic/character features. Special-token and chat-boilerplate features, such as the knowledge-cutoff date, sit isolated from the main component. In the Andromeda pirate-steering demo, the style flips near strength 0.5, pirate elements appear at 0.4, and factuality erodes from 0.6 (size wrong) to 1.0 (many facts wrong, rescaled to knots). At 1.4 every claim is false yet the Claude judge, scoring on a 0-100 'centiClaude' scale, still gives a high score. Moderation removed about 3.5% of the 70B features, versus about 30% for Llama 3.1 8B, counting dead features in both. Narrower, sparser mid-layer SAEs steer better, but this likely hurts classifier performance. The post argues that LLM-based steering evaluation is unreliable without factuality checks, that steering and classification need different SAE designs (crosscoders, Matryoshka SAEs may reconcile them), and that interpretability still needs a strong qualitative foundation.

What to take away

  1. 1. Goodfire trained SAEs on an intermediate layer of Llama 3.3 70B using LMSys-Chat-1M data and exposed them through an API, which it describes as the most capable openly available model with interpretability tooling.
  2. 2. In the Andromeda Galaxy pirate-steering demo, the judged score jumps around strength 0.5, where the style fully shifts, but pirate elements already appear at strength 0.4.
  3. 3. Factual accuracy degrades gradually under steering: at strength 0.6 most facts are correct except the galaxy's size, while at strength 1.0 many facts are wrong and rescaled to nautical units such as knots.
  4. 4. At steering strength 1.4 all factual claims are incorrect, yet the Claude judge still gives a high score on its 0-100 'centiClaude' scale, exposing a pitfall of LLM-based steering evaluation.
  5. 5. The UMAP feature map built with DataMapPlot shows semantically coherent clusters (biomedical, physics, programming, cognitive biases, name abstractions such as placeholders, citations and name prefixes, and a large phonetic/character cluster), even though the SAE was trained only on chat data.
  6. 6. Features for special formatting tokens and repetitive chat elements, such as the knowledge-cutoff date, appear as isolated points or small clusters away from the central component.
  7. 7. Moderation removed roughly 3.5% of features from the Llama 3.3 70B SAE versus roughly 30% from the Llama 3.1 8B SAE, with both figures including dead features, and mostly targeted graphic criminal content from the LMSys training data.
  8. 8. Narrower and sparser SAEs near the middle of the model steer better, whereas classification likely favors broader SAEs early or late in the model, so the two uses are in tension.
  9. 9. Open question: why do isolated features sit away from the central cluster, because of large special-token magnitudes or because the SAE and base model memorized frequently repeated chat content?
  10. 10. Open question: what mechanism makes latent steering damage factual recall, and do the phonetic/character features show feature absorption? Both are left unresolved.

Peer brief — for seminar discussion

Goodfire trained sparse autoencoders on an intermediate layer of Llama 3.3 70B using LMSys-Chat-1M chat data and released them through an API, together with an interactive UMAP feature map built with DataMapPlot. The map shows coherent clusters (biomedical, physics, programming, cognitive biases, name abstractions, and a large phonetic and character cluster), while special-token and chat-boilerplate features such as the knowledge-cutoff date sit apart from the main component. The post offers two explanations for the isolation: large special-token activation magnitudes, or memorization of highly repeated content. It does not test either. The load-bearing result is the steering demonstration. Steering on a pirate-style feature in an Andromeda Galaxy prompt, the style flips near strength 0.5, with traces at 0.4. Factuality then decays: the galaxy's size is wrong at 0.6, many facts are wrong and converted to nautical units by 1.0, and every claim is wrong at 1.4. The Claude-based judge, scored 0-100 in 'centiClaude' units, still rates the 1.4 output highly. The practical lesson is that single-axis LLM-judge steering evaluations can reward fluent style at the expense of truth, so factuality needs its own metric. The mechanism by which steering harms recall is unknown. The design implications are that narrower, sparser mid-layer SAEs steer better, while classification probably prefers broader SAEs early or late in the model. Crosscoders, Matryoshka SAEs, or iterative codebook feature learning might reconcile the two. Moderation removed about 3.5% of 70B features, compared with about 30% for Llama 3.1 8B, including dead features in both counts. A critical reader would push back first on evidence. The steering result is one prompt and one feature, judged by an LLM that the post itself shows to be flawed, and there are no reconstruction or loss-recovered numbers, no comparison to a baseline such as activation addition or difference-of-means steering, and no replication across features. The claim that the SAE learned 'surprisingly broad' concepts is based on browsing UMAP clusters, a projection that can distort distances. The post also concedes that it has not verified that the behavior-distinguishing latents have distinct effects. The headline that this is the most capable openly available model with interpretability tooling is a statement about release, not about scientific validity. The memorization hypothesis and the absorption question for phonetic features are the testable predictions to take away, and the narrower-is-better-for-steering claim is asserted without ablation, with a methodology post promised for later.

Methods (8)

Frameworks (3)

Datasets (1)

  • LMSYS-CHAT-1M
    Chat dataset (n=18,777 sampled) used to measure how much persona space PCs explain overall activation variance and to calibrate steering norms

Related work— refs + corpus + external arXiv

Cited / in-corpus / arXiv badges show which signals surfaced each row. Multi-source rows weighted higher.

Similar preprints — Semantic Scholar