finding
active
finding:arabic-feature-a-1-3450-and-b-1-1334-have-activation-correlation-of-0-91-across-40m-tokensArabic feature A/1/3450 and B/1/1334 have activation correlation of 0.91 across 40M tokens
Demonstrates universality of the Arabic script feature across two independently trained transformers
Source paper
extracted_from(2024) · Marc Carauleanu · Michael Vaiana · Judd Rosenblatt · Cameron Berg +1
Neighborhood — ranked by edge-count
Claims (1)
claim
- Authors take agnostic position on ontological status but universality evidence pushes toward features being real
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Universality of Hebrew script feature across two transformers
- Universality of base64 feature across two transformers
- Universality of DNA feature across two transformer models with different random seeds
- Pearson correlation of 0.74 between A/1/3450 activation and Arabic script proxy over 40M tokensfinding0.837Joint measure of sensitivity and specificity for the Arabic script feature
- Demonstrates activation specificity of the Arabic script sparse autoencoder feature
- Demonstrates that the Arabic feature is not aligned to any single neuron
- Demonstrates that activation similarity can diverge from logit weight similarity due to interference
- Systematic comparison showing features are substantially more universal than neurons across models