hypothesis
active
hypothesis:transcoders-could-cheat-jacobian-matching-via-large-weight-very-negative-bias-features-exploiting-the-l1-l0-gap

Transcoders could cheat Jacobian matching via large-weight, very negative-bias features exploiting the L1/L0 gap

Gradient masking risk; mitigated by tanh L1 or penalizing Jacobian contribution L1.

Source paper

extracted_from
A Toy Model of Mechanistic (Un)Faithfulness
Chris Olah

Neighborhood — ranked by edge-count

Methods (1)

method
  • Penalizing difference between transcoder and MLP Jacobians, computed cheaply via random vector products.