Earth Mover's Distance (EMD)

Primary quantitative measure of distributional divergence between natural and intervened representations

Neighborhood — ranked by edge-count

concept

Representational Divergence
about
Core phenomenon studied: when causal interventions shift internal representations away from the natural distribution

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Exponential Moving Average (EMA)concept0.747
Used to estimate gradients dynamically, with forgetting rate β.
DAS behavioral loss produces EMD along feature dimensions of 0.032±0.003 on synthetic 10-class datasetfinding0.714
Quantitative baseline for divergence using behavioral DAS loss on synthetic dataset
Representational divergence (as measured by EMD) can predict lower out-of-distribution intervention performanceclaim0.711
Practical utility of reducing divergence demonstrated through regression analysis
Average Euclidean cell movement distanceconcept0.694
Measure of how far cells travel during development; increased by stress sharing.
Modified CL loss produces EMD along feature dimensions of 0.007±0.001 on synthetic 10-class datasetfinding0.688
Quantitative improvement in divergence reduction using the modified CL loss on synthetic dataset
Exponential Moving Average Target Networkmethod0.687
Used in self-prior training as a slow target network for KL regularization
Mean difference patching on Llama-3-8B layer 10 produces intervened EMD exceeding the natural-natural baselinefinding0.685
Empirical demonstration that MDVP produces divergent representations in a real LLM
Hellinger distancemethod0.684
Metric used to define geometric space of output token probability distributions in behavior manifold analysis.