finding
active
finding:transcoders-easily-discover-the-relu-x-relu-x-solution-by-default-on-the-absolute-value-problemTranscoders easily discover the ReLU(x)+ReLU(-x) solution by default on the absolute value problem
Baseline result before repetition.
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Takeaway of toy experiments.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Baseline toy experiment.
- Toy computation y=abs(x) implemented by two ReLU neurons, used to study unfaithfulness.
- Without superposition.
- Synthetic theoretical example showing pernicious divergence via hidden pathway activation
- ReLU/JumpReLU restrict blocks to a cone and fail to recover superpositions a signed code resolves.claim0.744Justifies no coordinate-wise nonlinearity.
- Critique of activation-based interpretability methods.
- Second category of SAE/transcoder concerns, distinct from faithfulness.
- Core insight: reconstruction objective combined with appropriate initialization and KL regularization produces human-interpretable explanations as emergent property.