finding
active
finding:crosscoder-on-claude-3-sonnet-and-base-middle-layer-yields-three-groups-base-specific-finetuned-specific-shared-with-4-5-thousand-model-specific-features-each-of-1mCrosscoder on Claude 3 Sonnet and base middle layer yields three groups: base-specific, finetuned-specific, shared, with 4-5 thousand model-specific features each of 1M
Decomposition into shared and model-specific features.
Source paper
extracted_fromNeighborhood — ranked by edge-count
Claims (1)
claim
- Compared with base SAE transfer.
Datasets (1)
dataset
- Residual stream activations from the middle layer used for finetuning diff.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Multilingual features grow with scale.
- Family of count features.
- Cross-layer transcoder features form a mix of local, global and intermediate features by decoder normfinding0.790Preliminary decoder norm analysis.
- Preliminary observation.
- Specific application of SAE to extract features from the middle layer of Claude 3 Sonnet, at three scales (1M, 4M, 34M features).
- Validates Assumption D.3 that instruction-tuned models prefer representative distributions, supporting the VS theoretical framework
- Linked to Claude 3.5 Sonnet not exhibiting pro-animal-welfare preferences
- Decoder norm profile across 18 layers.