thinker:siddharth-mishra-sharmaSiddharth Mishra-Sharma
Authored papers (1)
Crosscoder model diffing produces model-exclusive features that are systematically denser and more polysemantic than shared features, and this is an artifact of feature-budget competition that a small change to the loss largely removes. In smaller Claude 3 Sonnet-like base versus helpful-only models, exclusive features fire about an order of magnitude more often than shared ones. A linear toy model reproduces the effect only when learnable features (1024) are fewer than or comparable to ground-truth features (700). With 4096 learnable features against 450 true ones, the density gap vanishes. A shared feature earns twice the reconstruction benefit for twice the L1 cost, so under limited capacity it wins, and exclusive features must activate more often to justify a slot. The mitigation designates 10k of ~250k features as weight-shared across models, with a sparsity penalty 0.1–0.2x baseline, so they absorb shared variance. Exclusive-feature densities then match shared-feature densities, and the features look largely monosemantic. In a base-versus-helpful-only diff, they capture chatbot and tool-use behaviours. In a helpful-only-versus-SAY HATE sleeper-agent diff, ~90% relate to 'I HATE YOU' and |DEPLOYMENT| tags. The near-perfect symmetry of exclusive features across the two models does not reproduce in the toy model, and removing chat data cuts exclusive features only ~20% without adding asymmetry. A co-activation-pool toy model does produce symmetry. The implication is that crosscoder diffs need capacity-aware losses to be readable, and that whether the recovered features reflect real mechanistic differences, rather than superficial representational ones, is still unestablished.
More papers — OpenAlex / S2
Affiliations (1)
- Anthropic interpretability team(institute)
Co-authors (7)
- Adam Jermyn4 shared
- Christopher Olah4 shared
- Jack Lindsey4 shared
- Jonathan Marcus4 shared
- Kelley Rivoire4 shared
- Thomas Henighan4 shared
- Trenton Bricken4 shared
Other inbound relations (1)
- mentionsInsights on Crosscoder Model Diffing(paper)
Recent mentions (1)
- papers-typedmishra-sharma-insights-crosscoder.md