finding
active
finding:hebrew-feature-a-1-416-and-b-1-1901-have-activation-correlation-of-0-92Hebrew feature A/1/416 and B/1/1901 have activation correlation of 0.92
Universality of Hebrew script feature across two transformers
Source paper
extracted_from(2024) · Marc Carauleanu · Michael Vaiana · Judd Rosenblatt · Cameron Berg +1
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Universality of DNA feature across two transformer models with different random seeds
- Arabic feature A/1/3450 and B/1/1334 have activation correlation of 0.91 across 40M tokensfinding0.863Demonstrates universality of the Arabic script feature across two independently trained transformers
- Universality of base64 feature across two transformers
- Systematic comparison showing features are substantially more universal than neurons across models
- Demonstrates that activation similarity can diverge from logit weight similarity due to interference
- Demonstrates specificity and sensitivity of DNA feature
- Hebrew feature is effectively invisible in the neuron basis
- Pearson correlation of 0.74 between A/1/3450 activation and Arabic script proxy over 40M tokensfinding0.757Joint measure of sensitivity and specificity for the Arabic script feature