question

active

question:when-it-is-not-okay-how-can-we-prevent-divergent-representations-from-occurring

When it is not okay, how can we prevent divergent representations from occurring?

Third core research question motivating the CL loss approach in Section 5

Source paper

extracted_from

Addressing divergent representations from causal interventions on neural networks

(2025) · Satchel Grant · Simon Jerome Han · Alexa R. Tartaglini · Christopher Potts

Neighborhood — ranked by edge-count

Claims (1)

claim

The CL auxiliary loss can directly reduce representational divergence in practical interpretability settings without sacrificing interpretability method accuracy
gates
Central practical contribution: the CL loss offers a viable mitigation strategy

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

When, and to what extent, is it okay for divergences to occur?question0.833
Second core research question motivating the theoretical analysis in Section 4
Divergent representations are a common, if not likely, outcome of causal interventions across a wide range of methodsclaim0.821
Core empirical claim of the paper supported by both theoretical proof and empirical demonstration
What if the concept being manipulated does not lie on a straight line in the model's representations?question0.786
The motivating question that opens the paper and leads to the development of manifold steering.
Do divergent representations change what an intervention can say about an NN's natural mechanisms?question0.781
Core research question motivating the paper
Patching h[1] with a divergent representation can activate distinct, hidden pathways that result in misleadingly confirmatory behavior and/or undetected behavior.quote0.777
Load-bearing description of the core pernicious divergence mechanism illustrated in Figure 1
Even if a case successfully meets all three criteria, this does not necessarily indicate that the corresponding sequence of representations is conscious. Rather, it suggests the observation of a potential 'consciousness' phenomenon within these representations — nothing more.quote0.770
Load-bearing epistemic caution the author places on the entire analytical framework.
Everything should be made as simple as possible, but not simpler.quote0.767
Einstein's assertion invoked to explain why BMR preserves accuracy while reducing complexity
The issue is not whether the process will produce living structure if done faithfully — it will — but whether the maker has the stamina and will to keep rejecting versions that lack life.claim0.765
Alexander's claim that the limiting factor in creating living structure is not method but the maker's persistence.