paper:costa-from-flat-to-hierarchical-extracting-spa-2025From flat to hierarchical: Extracting sparse representations with matching pursuit
Related work— refs + corpus + external arXiv
Cited / in-corpus / arXiv badges show which signals surfaced each row. Multi-source rows weighted higher.
- Anomaly Detection in Hierarchical Data Streams under Unknown ModelsQing Zhao, Chang Liu, Chen-Nee Chuah Sattar Vakili2017≈ 75%
- ShaRP: Shape-Regularized Multidimensional ProjectionsAlister Machado and Alexandru Telea and Michael Behrisch2026≈ 73%
- ≈ 73%
- From Atoms to Trees: Building a Structured Feature Forest with Hierarchical Sparse AutoencodersYang Zhan, Jiedong Jiang, Tianyang Liu, Mingrui Wu, Zhennan Zhou, Bin Dong Yifan Luo2026≈ 73%
- scShapeBench: Discovering geometry from high dimensional scRNAseq dataJo\~ao Felipe Rocha, Brian Tshilengi Di Bassinga, Zachary Warren, Matthew Scicluna, C\'esar Miguel Valdez C\'ordova, Shabarni Gupta, Leire Torices, Daniel Neumann, Timothy J. Mann, Ihuan Gunawan, Dhananjay Bhaskar, John G Lock, Christine L Chaffer, Guy Wolf, Smita Krishnaswamy Andrew J Steindl2026≈ 73%
- ≈ 73%
- SparseRM: A Lightweight Preference Modeling with Sparse AutoencoderJiahao Li, Zheren Fu, Yi Tu, Jiajun Li, Zhendong Mao, Yongdong Zhang Dengcan Liu2025≈ 73%
- Train One Sparse Autoencoder Across Multiple Sparsity Budgets to Preserve Interpretability and AccuracyYaroslav Aksenov, Daniil Laptev, Vadim Kurochkin, Gleb Gerasimov, Nikita Koryagin, Daniil Gavrilov Nikita Balagansky2025≈ 72%
- Practitioner Motives to Use Different Hyperparameter Optimization MethodsNiklas Hasebrook, Felix Morsbach, Marc-Andr\'e Z\"oller, J\"org Franke, Marius Lindauer, Frank Hutter, Ali Sunyaev Niclas Kannengie{\ss}er2026≈ 72%
- Algorithm Selection with Probing Trajectories: Benchmarking the Choice of Classifier ModelQuentin Renau and Emma Hart2025≈ 72%
- A Pattern Language for Machine Learning TasksIan Fan, Tuomas Laakkonen, Neil John Ortega, Thomas Hoffmann, Vincent Wang-Mascianica Benjamin Rodatz2025≈ 72%
- Inverse classification with logistic and softmax classifiers: efficient optimizationMiguel \'A. Carreira-Perpi\~n\'an and Suryabhan Singh Hada2026≈ 72%
- Improving Sparse Autoencoder with Dynamic AttentionJinsen Zhang, Dawei Su, Hui Huang Dongsheng Wang2026≈ 72%
- Smoothing the Edges: Smooth Optimization for Sparse Regularization using Hadamard OverparametrizationChris Kolb and Christian L. M\"uller and Bernd Bischl and David R\"ugamer2026≈ 72%
- A Unified Theory of Sparse Dictionary Learning in Mechanistic Interpretability: Piecewise Biconvexity and Spurious MinimaHarshvardhan Saini, Zhaoqian Yao, Zheng Lin, Yizhen Liao, Jingyi Cui, Yisen Wang, Mengnan Du, Dianbo Liu Yiming Tang2026≈ 72%
- ≈ 71%
- ≈ 70%
- Mechanistic Knobs in LLMs: Retrieving and Steering High-Order Semantic Features via Sparse Autoencodersin corpus2026≈ 69%
- Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representationsin corpus2023≈ 69%
- Toy Models of Superpositionin corpus2022≈ 69%
- Model Alignment Searchin corpus2025≈ 68%
- ≈ 68%
- ≈ 68%
- ≈ 68%
- Open Problems in Mechanistic Interpretabilityin corpus2025≈ 68%
- ≈ 68%
- ≈ 67%
- ≈ 67%
Similar preprints — Semantic Scholar
Cited by (6)
- Understanding sparse autoencoder scaling in the presence of feature manifolds
Sparse autoencoder scaling laws can break down when activations contain feature manifolds: if per-feature loss falls slowly with latent count, an SAE spends its latents tiling common manifolds instead
- Do Sparse Autoencoders Capture Concept Manifolds?
Sparse autoencoders trained on Llama3.1-8B (layer 19, 500M Pile tokens, 25 SAEs across Standard, JumpReLU, TopK, BatchTopK and Matryoshka architectures at VE > 0.85) do not capture continuous concept
- Structuring Sparsity: Block-Sparse Featurizers Capture Visual Concept Manifolds
Lifting the featurizer's atom from a direction to a block of directions recovers concept manifolds that SAEs shatter, and block sparsity is the MAP-matched prior for a representation that is a sparse
- When Models Manipulate Manifolds: The Geometry of a Counting Task
Claude 3.5 Haiku's linebreaking in fixed-width text is implemented by manipulating low-dimensional, rippled 1-D feature manifolds: character count, line width, characters remaining and next-token leng
- Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behavior
Manifold steering — intervening on model activations along paths constrained to lie on a learned activation manifold M_h rather than along Euclidean linear directions — produces behavioral trajectorie
- Arithmetic in the Wild: Llama uses Base-10 Addition to Reason About Cyclic Concepts
Llama-3.1-8B solves cyclic arithmetic (e.g., "what month is six months after August?") not by performing modular addition in the period of the cyclic concept (12 for months, 7 for days of the week) as