artifact
active
artifact:goodfire-llama-3-8b-instruct-layer-19-saeGoodfire Llama-3-8B-Instruct layer-19 SAE
ReLU SAE trained on residual stream layer 19 with MSE+L1 loss.
Neighborhood — ranked by edge-count
Papers (1)
paper
- Understanding and Steering Llama 3 with Sparse Autoencoders - Goodfireintroducesmentions
Methods (3)
method
- Mean number of nonzero features as sparsity metric, tracked as EMA.
- Buffer, shuffle, and write activations to avoid autocorrelation between batches.
- Log-loss recovery ratio between SAE-inserted and ablated model.
Artifacts (1)
artifact
- Llama-3-8B-InstructaboutInstruction-tuned model analyzed.
Datasets (1)
dataset
- LMSYS-CHAT-1MusesChat dataset (n=18,777 sampled) used to measure how much persona space PCs explain overall activation variance and to calibrate steering norms