finding
active
finding:base64-feature-a-1-2357-and-b-1-2165-have-activation-correlation-of-0-85Base64 feature A/1/2357 and B/1/2165 have activation correlation of 0.85
Universality of base64 feature across two transformers
Source paper
extracted_from(2024) · Marc Carauleanu · Michael Vaiana · Judd Rosenblatt · Cameron Berg +1
Neighborhood — ranked by edge-count
Claims (1)
claim
- Authors take agnostic position on ontological status but universality evidence pushes toward features being real
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Universality of DNA feature across two transformer models with different random seeds
- Universality of Hebrew script feature across two transformers
- Arabic feature A/1/3450 and B/1/1334 have activation correlation of 0.91 across 40M tokensfinding0.854Demonstrates universality of the Arabic script feature across two independently trained transformers
- Causal validation of base64 feature function via pinned feature sampling
- Shows base64 feature is polysemantic at neuron level but monosemantic as learned feature
- Demonstrates that activation similarity can diverge from logit weight similarity due to interference
- Systematic comparison showing features are substantially more universal than neurons across models
- Demonstrates specificity and sensitivity of DNA feature