thinker:thomas-jiralerspongThomas Jiralerspong
Authored papers (1)
Crosscoders can diff models with different architectures, and a modified crosscoder, the Dedicated Feature Crosscoder (DFC), surfaces behaviorally meaningful model-exclusive features without supervision. The DFC hard-partitions the 131,072-feature dictionary into Model A-exclusive, Model B-exclusive and shared sets (5% exclusive each, about 6,550 features), so exclusive features have decoder norm exactly zero in the other model and receive no gradient from its reconstruction. This removes the standard crosscoder's prior toward shared features (relative norm near 0.5). On Llama-3.1-8B-Instruct vs Qwen3-8B, the DFC found a Qwen CCP-alignment feature and a Llama American-exceptionalism feature. On GPT-OSS-20B vs DeepSeek-R1-0528-Qwen3-8B, it found a GPT-OSS copyright-refusal feature, a ChatGPT-identity feature, and a second CCP-alignment feature in DeepSeek. Steering each feature across 30 prompts moves behavior in its home model and barely moves the other. Each reached the maximum score of 5 on the paper's stitching-based exclusivity score, which the paper introduces as an independent check on exclusivity. In a toy model with 2,048 ground-truth concepts, the DFC recovers more exclusive concepts than standard and Designated Shared Feature (DSF) crosscoders, at the cost of more false positives. Fraction of variance explained is unchanged (0.817 vs 0.817). Most exclusive features are not behaviorally meaningful, and the granular pro-China features appeared in only 2-3 of 5 runs. The paper argues that cross-architecture crosscoder diffing is a high-recall screen-and-verify tool for finding unknown unknowns in new models, to be followed by causal steering validation.
More papers — OpenAlex / S2
Affiliations (2)
- Mila / Université de Montréal(institute)
- Anthropic(institute)
Co-authors (1)
- Trenton Bricken9 shared
Other inbound relations (1)
Recent mentions (1)
- papers-typedjiralerspong-2026-cross-architecture.md