finding
active
finding:absolute-value-model-n-3-m-6-uses-two-neurons-per-feature-relu-x-and-relu-xAbsolute value model (n=3, m=6) uses two neurons per feature: ReLU(x) and ReLU(-x)
Without superposition.
Neighborhood — ranked by edge-count
Papers (1)
paper
- Toy Models of Superpositionmentions
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Absolute value model (n=100, m=40) computes abs in superposition when features are sparsefinding0.822Computation in superposition.
- Mechanistic motif.
- Mirrors real networks.
- Transcoders easily discover the ReLU(x)+ReLU(-x) solution by default on the absolute value problemfinding0.782Baseline result before repetition.
- ReLU hidden layer model (n=10, m=5) shifts from monosemantic to polysemantic neurons as sparsity increasesfinding0.768Neuron-level phase change.
- Core demonstration.
- Synthetic theoretical example showing pernicious divergence via hidden pathway activation
- Working definition adopted by the paper.