paper:mcgrath-mapping-latent-95c0a3Mapping the Latent Space of Llama 3.3 70B - Goodfire
TL;DR
Sparse autoencoders trained on an intermediate layer of Llama 3.3 70B, using LMSys-Chat-1M data, yield a browsable, steerable feature space, and the clearest results are about where it breaks: steering works but silently degrades facts, and LLM-judged scores fail to detect this. Goodfire releases the SAEs through an API and maps the features with a DataMapPlot UMAP visualization, which reveals clusters for biomedical knowledge, physics, programming, cognitive biases, name abstractions, and phonetic/character features. Special-token and chat-boilerplate features, such as the knowledge-cutoff date, sit isolated from the main component. In the Andromeda pirate-steering demo, the style flips near strength 0.5, pirate elements appear at 0.4, and factuality erodes from 0.6 (size wrong) to 1.0 (many facts wrong, rescaled to knots). At 1.4 every claim is false yet the Claude judge, scoring on a 0-100 'centiClaude' scale, still gives a high score. Moderation removed about 3.5% of the 70B features, versus about 30% for Llama 3.1 8B, counting dead features in both. Narrower, sparser mid-layer SAEs steer better, but this likely hurts classifier performance. The post argues that LLM-based steering evaluation is unreliable without factuality checks, that steering and classification need different SAE designs (crosscoders, Matryoshka SAEs may reconcile them), and that interpretability still needs a strong qualitative foundation.
What to take away
- 1. Goodfire trained SAEs on an intermediate layer of Llama 3.3 70B using LMSys-Chat-1M data and exposed them through an API, which it describes as the most capable openly available model with interpretability tooling.
- 2. In the Andromeda Galaxy pirate-steering demo, the judged score jumps around strength 0.5, where the style fully shifts, but pirate elements already appear at strength 0.4.
- 3. Factual accuracy degrades gradually under steering: at strength 0.6 most facts are correct except the galaxy's size, while at strength 1.0 many facts are wrong and rescaled to nautical units such as knots.
- 4. At steering strength 1.4 all factual claims are incorrect, yet the Claude judge still gives a high score on its 0-100 'centiClaude' scale, exposing a pitfall of LLM-based steering evaluation.
- 5. The UMAP feature map built with DataMapPlot shows semantically coherent clusters (biomedical, physics, programming, cognitive biases, name abstractions such as placeholders, citations and name prefixes, and a large phonetic/character cluster), even though the SAE was trained only on chat data.
- 6. Features for special formatting tokens and repetitive chat elements, such as the knowledge-cutoff date, appear as isolated points or small clusters away from the central component.
- 7. Moderation removed roughly 3.5% of features from the Llama 3.3 70B SAE versus roughly 30% from the Llama 3.1 8B SAE, with both figures including dead features, and mostly targeted graphic criminal content from the LMSys training data.
- 8. Narrower and sparser SAEs near the middle of the model steer better, whereas classification likely favors broader SAEs early or late in the model, so the two uses are in tension.
- 9. Open question: why do isolated features sit away from the central cluster, because of large special-token magnitudes or because the SAE and base model memorized frequently repeated chat content?
- 10. Open question: what mechanism makes latent steering damage factual recall, and do the phonetic/character features show feature absorption? Both are left unresolved.
Peer brief — for seminar discussion
Goodfire trained sparse autoencoders on an intermediate layer of Llama 3.3 70B using LMSys-Chat-1M chat data and released them through an API, together with an interactive UMAP feature map built with DataMapPlot. The map shows coherent clusters (biomedical, physics, programming, cognitive biases, name abstractions, and a large phonetic and character cluster), while special-token and chat-boilerplate features such as the knowledge-cutoff date sit apart from the main component. The post offers two explanations for the isolation: large special-token activation magnitudes, or memorization of highly repeated content. It does not test either. The load-bearing result is the steering demonstration. Steering on a pirate-style feature in an Andromeda Galaxy prompt, the style flips near strength 0.5, with traces at 0.4. Factuality then decays: the galaxy's size is wrong at 0.6, many facts are wrong and converted to nautical units by 1.0, and every claim is wrong at 1.4. The Claude-based judge, scored 0-100 in 'centiClaude' units, still rates the 1.4 output highly. The practical lesson is that single-axis LLM-judge steering evaluations can reward fluent style at the expense of truth, so factuality needs its own metric. The mechanism by which steering harms recall is unknown. The design implications are that narrower, sparser mid-layer SAEs steer better, while classification probably prefers broader SAEs early or late in the model. Crosscoders, Matryoshka SAEs, or iterative codebook feature learning might reconcile the two. Moderation removed about 3.5% of 70B features, compared with about 30% for Llama 3.1 8B, including dead features in both counts. A critical reader would push back first on evidence. The steering result is one prompt and one feature, judged by an LLM that the post itself shows to be flawed, and there are no reconstruction or loss-recovered numbers, no comparison to a baseline such as activation addition or difference-of-means steering, and no replication across features. The claim that the SAE learned 'surprisingly broad' concepts is based on browsing UMAP clusters, a projection that can distort distances. The post also concedes that it has not verified that the behavior-distinguishing latents have distinct effects. The headline that this is the most capable openly available model with interpretability tooling is a statement about release, not about scientific validity. The memorization hypothesis and the absorption question for phonetic features are the testable predictions to take away, and the narrower-is-better-for-steering claim is asserted without ablation, with a methodology post promised for later.
Methods (8)
- AutoSteerFunctionality that automatically finds SAE latents for a desired behaviour and sets their weights.
- DataMapPlotTool used to create the interactive feature map.
- Feature steering (clamping feature activations)Modifying model behavior by clamping SAE feature activations to specific values during forward pass.
- Harmful feature moderationRemoval of harmful and dead features from released SAEs.
- Iterative Codebook Feature LearningFlexible decoding technique mentioned as possible remedy.
- LLM-based steering evaluation pipelineClaude rates coherence and behavior on 0-100 scale, called centiClaude.
- Sparse Autoencoders (SAEs)Unsupervised baseline method for decomposing activations into sparse linear combinations; NLAs positioned as complementary approach.
- UMAPUsed to visualize models in 2D space based on representational distance
Frameworks (3)
- CrosscodersLearn a shared feature space across two models; basis of the work.
- Matryoshka SAEsNested dictionary approach that inspires the nested loss.
- Responsible Scaling PlanPlanned framework built on safety evals.
Datasets (1)
- LMSYS-CHAT-1MChat dataset (n=18,777 sampled) used to measure how much persona space PCs explain overall activation variance and to calibrate steering norms
Findings (8)
- SAE trained on internet chat data learned clusters for cognitive biases, biomedical knowledge, physics, programming, name abstractions, and phonetic/character features
Feature cluster examples from the UMAP.
- Features for special formatting tokens and repetitive chat elements appear as isolated points or small clusters away from the central UMAP component
Observation on the Llama 3.3 70B SAE feature map.
- Factuality degrades with steering strength: at 0.6 only galaxy size is wrong, at 1.0 many facts incorrect and rescaled to nautical units
Pirate steering demo.
- Steering Llama 3.3 70B toward pirate speech: evaluation jumps sharply near strength 0.5, with some pirate elements already at 0.4
Andromeda Galaxy steering demo.
- Approximately 30% of Llama 3.1 8B SAE features and 3.5% of Llama 3.3 70B SAE features were removed, including dead features
Moderation statistics.
- At steering strength 1.4 all factual claims are incorrect yet the LLM evaluator still gives a high score
Pitfall of LLM-based steering evaluation.
- Name-related abstractions (placeholders, citations, name prefixes) cluster together
Specific cluster observed.
- Large and detailed cluster of phonetic and character-related features
Specific cluster observed.
Claims (8)
- Feature steering benefits from narrow sparse mid-model SAEs whereas classification benefits from broader SAEs early or late in the model
Limitations section.
- Narrower and sparser SAEs than conventionally trained are better for steering, but in tension with classifier performance
Authors' experience-based claim.
- Current SAEs capture only a fraction of model computation due to limited training data and incomplete reconstruction
Limitation statement.
- Llama 3.3 70B is, to the authors' knowledge, the most capable openly available model with interpretability tooling
Positioning claim of the release.
- Making interpretability tools easily available on a powerful model will enable both new research and new products
Motivation for the release.
- Open-sourcing SAEs on current leading open-source models does not meaningfully affect biorisk or persuasion risk
Risk judgment because base model is insufficiently capable.
- Harmful features mostly reflect graphic criminal content subsets of the LMSys chat training data
Explanation for moderated features.
- Interpretability and steering still need a strong qualitative foundation
Caveat after the evaluator pitfall.
Hypotheses (3)
- A combination of crosscoders and flexible decoding (Matryoshka, iterative codebook) could resolve the steering-classification tension
Proposed remedy.
- Special tokens have very large magnitude, so their SAE features also have large magnitude and sit far from other points
First explanation for isolated features.
- Frequently repeated chat features (e.g. knowledge cutoff) are memorized by the SAE and base model, causing isolation
Second explanation for isolated features.
Questions (4)
- Is incomplete SAE reconstruction due to a fundamental architecture limitation (e.g. truly nonlinear features) or a mundane cause?
Open question in limitations.
- Do latents for precise distinctions in types of behaviour have distinct steering effects?
Authors have not verified this.
- What is the mechanism by which factual recall is damaged by latent steering?
Explicitly unknown mechanism.
- Do the phonetic and character-related features exhibit absorption?
Open question about feature cluster.
Related work— refs + corpus + external arXiv
Cited / in-corpus / arXiv badges show which signals surfaced each row. Multi-source rows weighted higher.
- ≈ 91%
- CorrSteer: Generation-Time LLM Steering via Correlated Sparse Autoencoder FeaturesZekun Wu, Adriano Koshiyama Seonglae Cho2026≈ 84%
- LF-Steering: Latent Feature Activation Steering for Enhancing Semantic Consistency in Large Language ModelsRongjun Li, Weixuan Wang, Ziyu Zhou, Zhiyong Feng, Wei Peng Jingyuan Yang2025≈ 84%
- ≈ 84%
- Interpretable and Steerable Concept Bottleneck Sparse AutoencodersTsui-Wei Weng, Vivek Narayanaswamy, Shusen Liu, Wesam A. Sakla, Kowshik Thopalli Akshay Kulkarni2026≈ 84%
- Steering Conceptual Bias via Transformer Latent-Subspace ActivationVansh Sharma and Venkat Raman2025≈ 83%
- The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetsin corpus2023≈ 83%
- ≈ 83%
- Steering Knowledge Selection Behaviours in LLMs via SAE-Based Representation EngineeringAlessio Devoto, Giwon Hong, Xiaotang Du, Aryo Pradipta Gema, Hongru Wang, Xuanli He, Kam-Fai Wong, Pasquale Minervini Yu Zhao2025≈ 82%
- Measuring and Guiding MonosemanticityFelix Friedrich, Manuel Brack, Stephan W\"aldchen, Bj\"orn Deiseroth, Patrick Schramowski, Kristian Kersting Ruben H\"arle2025≈ 82%
- Mechanistic Indicators of Steering Effectiveness in Large Language ModelsHao Xue, Flora Salim Mehdi Jafari2026≈ 82%
- Activation Steering for Aligned Open-ended Generation without Sacrificing CoherenceMartin Zborowski, Alberto Tosato, Gauthier Gidel, Tommaso Tosato Niklas Herbster2026≈ 82%
- Enhancing LLM Steering through Sparse Autoencoder-Based Vector RefinementXuansheng Wu, Dong Shu, Yunpu Ma, Ninghao Liu Anyi Wang2025≈ 82%
- Interpretable LLM Guardrails via Sparse Representation SteeringZhibo Wang, Huiyu Xu, Hejun Lin, Wenhui Zhang, Zhixuan Chu Zeqing He2025≈ 82%
- FineScope : SAE-guided Data Selection Enables Domain Specific LLM Pruning and FinetuningChaitali Bhattacharyya and Hyunsei Lee and Junyoung Lee and Shinhyoung Jang and Il hong Suh and Yeseong Kim2026≈ 82%
- ≈ 82%
- When the Coffee Feature Activates on Coffins: An Analysis of Feature Extraction and Steering for Mechanistic InterpretabilityMarkus Maier, Frederick Eberhardt Raphael Ronge2026≈ 82%
- Controllable LLM Reasoning via Sparse Autoencoder-Based SteeringWenjie Wang, Mingfeng Xue, Boyi Deng, Fengli Xu, Dayiheng Liu, Fuli Feng Yi Fang2026≈ 82%
- A Comparative Analysis of Sparse Autoencoder and Activation Difference in Language Model SteeringJiaqing Xie2025≈ 82%
- Anatomy of Post-Training: Using Interpretability to Characterize Data and Shape the Learning Signalin corpus2026≈ 82%
- YaPO: Learnable Sparse Activation Steering Vectors for Domain AdaptationRania Hossam Elmohamady Elbadry, Hadi Abdine, Preslav Nakov, Michalis Vazirgiannis, Guokan Shang Abdelaziz Bounhar2026≈ 82%
- Mechanistic Knobs in LLMs: Retrieving and Steering High-Order Semantic Features via Sparse Autoencodersin corpus2026≈ 82%
- ≈ 81%
- Testing the Limits of Truth Directions in LLMsin corpus2026≈ 81%
- Do Sparse Autoencoders Capture Concept Manifolds?in corpus2026≈ 81%
- ≈ 81%
- Probe-Based Data Attribution: Surfacing and Mitigating Undesirable Behaviors in LLM Post-Trainingin corpus2026≈ 81%
- Cross-Architecture Model Diffing with Crosscoders: Unsupervised Discovery of Differences Between LLMsin corpus2026≈ 81%