thinker:lewis-owenLewis, Owen
Authored papers (2)
Superposition causally contributes to adversarial vulnerability, and the link runs in both directions: in a toy ReLU autoencoder (n=100 features, m=20 hidden units), raising feature sparsity raises features-per-dimension and adversarial vulnerability (r≈0.99 under Elhage-style attacks), and adversarial training with α=0.5, ε=0.1‖x‖₂ for 150,000 steps lowers features-per-dimension at matched sparsity. The same direction holds in real networks: robust ImageNet ResNet18s from Salman et al. (2020) trained at L2 ε∈{0,1,3,5} give lower SAE reconstruction MSE at fixed TopK sparsity (k=8,16,32), read as a proxy for less superposition. Gorton and Lewis supply the evidence by pairing a toy-model setup in which superposition is exactly measurable and controllable via sparsity with SAE loss as a proxy in ResNet18. Geometry supports the mechanism: non-robust toy models show about 2× the mean off-diagonal interference (WᵀW) of robust ones, and PGD attacks raise SAE L0 at layer4 from 126 to 351 (2.8×), whereas FGSM barely moves it (1.02×). Robust models also look more robust than their superposition level predicts, which suggests interference structure matters beyond its total amount. The causal claim in real models is only one-directional, because superposition cannot be intervened on there. The paper argues superposition is at least one causal factor and possibly the primary one. If so, robustness trades off against the capacity superposition provides, and interpretability and robustness research share objectives.
More papers — OpenAlex / S2
Co-authors (12)
- Bushnaq, Lucius4 shared
- Gorton, Liv4 shared
- Merullo, Jack4 shared
- Owen Lewis4 shared
- Vatsavaya, Srihita4 shared
- Aleksander Madry2 shared
- David Yunis2 shared
- Ian Goodfellow2 shared
- Jack Merullo2 shared
- James Martens2 shared
- Liv Gorton2 shared
- Lucius Bushnaq2 shared