finding
active
finding:adding-jacobian-matching-loss-dramatically-reduces-datapoint-feature-memorization-metricAdding Jacobian matching loss dramatically reduces datapoint-feature memorization metric
Result of Jacobian matching.
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Positive result with caveats.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Toy-model fix of unfaithfulness.
- Motivates Jacobian connection.
- Transcoders could cheat Jacobian matching via large-weight, very negative-bias features exploiting the L1/L0 gaphypothesis0.737Gradient masking risk; mitigated by tanh L1 or penalizing Jacobian contribution L1.
- Penalizing difference between transcoder and MLP Jacobians, computed cheaply via random vector products.
- Ethical implication about the nature of AI training experience if the thesis holds
- Hyperparameter scan result.
- Interpretation of illusion.
- Core theoretical claim about the target of representation learning