method
active
method:absolute-value-transcoder-problemAbsolute value transcoder problem
Toy computation y=abs(x) implemented by two ReLU neurons, used to study unfaithfulness.
Neighborhood — ranked by edge-count
Papers (1)
paper
- A Toy Model of Mechanistic (Un)Faithfulnessintroducesmentions
Methods (1)
method
- TranscodersaboutDecomposition method for activations; VPD is compared against transcoders in sparsity-reconstruction tradeoff.
Conceptual bridges
2-hop · via this method's ideasWhere ideas in this method connect to the rest of the corpus — the same concept, an analogy, or a restatement elsewhere.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Transcoders easily discover the ReLU(x)+ReLU(-x) solution by default on the absolute value problemfinding0.813Baseline result before repetition.
- Uses activations from one layer to predict the next; special case of crosscoders.
- Second category of SAE/transcoder concerns, distinct from faithfulness.
- Used to express MLP in monosemantic features, 4096 features with JumpReLU.
- Baseline toy experiment.
- Quantifies certainty in beliefs; modulates policy selection; related to dopamine encoding.
- Decomposition approach mixing frozen base attention patterns.
- Neural network architecture that learns compressed representations; SOHMs are functionally equivalent.