finding
active
finding:allocating-10k-of-250k-features-as-shared-with-0-1-0-2x-sparsity-penalty-makes-exclusive-feature-densities-similar-to-shared-ones-while-remaining-symmetric

Allocating 10k of ~250k features as shared with 0.1-0.2x sparsity penalty makes exclusive feature densities similar to shared ones, while remaining symmetric

Mitigation works on real models.

Source paper

extracted_from
Insights on Crosscoder Model Diffing
Siddharth Mishra-Sharma · Trenton Bricken · Jack Lindsey · Adam Jermyn +4

Neighborhood — ranked by edge-count

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.