claim
active
claim:jacobian-matching-can-solve-mechanistic-faithfulness-in-this-toy-exampleJacobian matching can solve mechanistic faithfulness in this toy example
Positive result with caveats.
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (2)
finding
- Result of Jacobian matching.
- Toy-model fix of unfaithfulness.
Claims (1)
claim
- Limit on Jacobian matching.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Penalizing difference between transcoder and MLP Jacobians, computed cheaply via random vector products.
- Future work.
- Takeaway of toy experiments.
- Transcoders could cheat Jacobian matching via large-weight, very negative-bias features exploiting the L1/L0 gaphypothesis0.768Gradient masking risk; mitigated by tanh L1 or penalizing Jacobian contribution L1.
- Symmetry is not explained by the base toy model.
- Practical implication.
- For reasoning faithfulness, larger models may behave worse than smaller ones because they can predict answers without CoT.hypothesis0.741Motivates §3.
- Figure 4.