thinker
active
thinker:wattenberg-martin

Wattenberg, Martin

Authored
1
Introduces
0
Studies
0
Affiliations
0
Cited by
5

Authored papers (1)

  • Sparse autoencoders carry a hidden i.i.d.-over-time prior that conflicts with how language model activations behave, and replacing it with a temporal prior recovers structure SAEs miss. Rewriting the SAE objective as MAP estimation shows the sparsity penalties of ReLU, TopK, BatchTopK and JumpReLU SAEs are additive over tokens (Prop. 4.1). They therefore assume concepts are independent across time and that sparsity is time-invariant. Gemma-2-2B and Llama-3.1-8B activations contradict this. Intrinsic dimension, measured with a Gram-matrix U-statistic, grows with sequence position, while a permuted stationary surrogate stays flat. Up to 80% of a token's variance is explained by a 500-token context. Temporal Feature Analysis (TFA) splits each activation into a predictable component, projected onto past context through attention, and a sparse novel residual assumed i.i.d. At Gemma-2-2B layer 12 it matches SAEs on NMSE (0.139 vs 0.139 for BatchTopK on web text). Its predictive code recovers event boundaries in 50 GPT-5 stories, keeps garden-path parses stable, and recovers the graph geometry in the Park et al. in-context graph task. It supports counterfactual edits on 4/9 nodes, where no SAE succeeds. On Ultrachat it also lowers tortuosity from about 25–30 to 2.0. Interpretability tools need inductive biases matched to the data's temporal generative structure, and features are better treated as evolving low-dimensional manifolds than as isolated directions.

More papers — OpenAlex / S2