framework
active
framework:designated-shared-feature-dsf-crosscoderDesignated Shared Feature (DSF) Crosscoder
Baseline from Mishra-Sharma et al. dedicating features to shared variance.
Neighborhood — ranked by edge-count
Papers (2)
paper
- Insights on Crosscoder Model Diffingintroduces
Frameworks (2)
framework
- Dedicated Feature Crosscoder (DFC)related_toArchitectural variant partitioning the dictionary into A-exclusive, B-exclusive and shared sets.
- CrosscodersextendsLearn a shared feature space across two models; basis of the work.
Findings (1)
finding
- Hyperparameter sweep.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Motivation for DFC.
- Features with similar decoder magnitude across both models.
- Cross-layer transcoder features form a mix of local, global and intermediate features by decoder normfinding0.769Preliminary decoder norm analysis.
- Each feature assigned a window of K layers to reduce training FLOPs.
- Train a single SAE encoding and decoding activations from two models simultaneously to find shared and exclusive features.
- Sparse dictionary learning variant that reads and writes to multiple layers or models, yielding shared features.
- MLP replacement with sparse features underlying the original attribution graphs.
- Proposed variation designating ~10k of ~250k features as shared with penalty 0.1-0.2x baseline to soak up shared variance.