hypothesis
active
hypothesis:transcoders-could-cheat-jacobian-matching-via-large-weight-very-negative-bias-features-exploiting-the-l1-l0-gapTranscoders could cheat Jacobian matching via large-weight, very negative-bias features exploiting the L1/L0 gap
Gradient masking risk; mitigated by tanh L1 or penalizing Jacobian contribution L1.
Neighborhood — ranked by edge-count
Papers (1)
paper
Methods (1)
method
- Jacobian matchingcitesPenalizing difference between transcoder and MLP Jacobians, computed cheaply via random vector products.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Core motivation.
- Baseline toy experiment.
- Toy-model fix of unfaithfulness.
- Positive result with caveats.
- Limit on Jacobian matching.
- Motivating problem.
- Demonstrates that early-layer probes capture sentence polarity rather than truth.
- Circuit simplification theory.