claim
active
claim:a-simple-relu-l1-sae-was-chosen-for-simplicity-and-velocity-because-topk-and-jumprelu-may-have-high-frequency-featuresA simple ReLU/L1 SAE was chosen for simplicity and velocity because TopK and JumpReLU may have high-frequency features
Design justification.
Source paper
extracted_fromNeighborhood — ranked by edge-count
Papers (1)
paper
Questions (1)
question
- Future topic.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Trained on toy manifolds.
- SAE variant trained on ResNet18.
- ReLU underperforms.
- Training dynamics.
- ReLU/JumpReLU restrict blocks to a cone and fail to recover superpositions a signed code resolves.claim0.761Justifies no coordinate-wise nonlinearity.
- Baseline SAE architecture.
- Exception to ReLU underperformance.
- Evidence in real models.