paper
referenced-only
2018
paper:arxiv-1806-08887

The Sparse Manifold Transform

ByYu-Bei Chen·Dylan M. Paiton·B. Olshausen
Original abstract (expand)

We present a signal representation framework called the sparse manifold transform that combines key ideas from sparse coding, manifold learning, and slow feature analysis. It turns non-linear transformations in the primary sensory signal space into linear interpolations in a representational embedding space while maintaining approximate invertibility. The sparse manifold transform is an unsupervised and generative framework that explicitly and simultaneously models the sparse discreteness and low-dimensional manifold structure found in natural scenes. When stacked, it also models hierarchical composition. We provide a theoretical description of the transform and demonstrate properties of the learned representation on both synthetic data and natural videos.

Similar preprints — Semantic Scholar

Cited by (1)

  • Do Sparse Autoencoders Capture Concept Manifolds?

    Sparse autoencoders trained on Llama3.1-8B (layer 19, 500M Pile tokens, 25 SAEs across Standard, JumpReLU, TopK, BatchTopK and Matryoshka architectures at VE > 0.85) do not capture continuous concept