claim

active

claim:ei-and-normalized-ei-could-serve-as-a-unified-metric-for-out-of-distribution-generalization

EI and normalized EI could serve as a unified metric for out-of-distribution generalization.

Conjecture that maximizing EI yields causal representations invariant to distribution shifts.

Source paper

extracted_from

Emergence and Causality in Complex Systems: A Survey on Causal Emergence and Related Quantitative Studies

(2023) · Bing Yuan · Jiang Zhang · Aobo Lyu · Jiayun Wu +5

Neighborhood — ranked by edge-count

Findings (1)

finding

NIS+ outperforms NIS, variational autoencoders, and feed-forward neural networks in out-of-distribution generalization experiments.
supports
Yang et al. (2023) result linking EI maximization to robust generalization.

Communities (3)

community

Causal emergence in biological systems
members_of
Examines how macro-scale causal power exceeds micro-scale in living and learning systems.
Causal emergence via information-geometric coarse-graining
members_of
Framework using effective information (EI) and NIS+ to automatically discover macro-scale dynamics from micro-level data, validated on fMRI, Conway's Game of Life, and SIR models.
Causal emergence & effective information
members_of
Using EI/normalized EI to evaluate macro-scale causal structure and coarse-graining quality.

Concepts (1)

concept

Out-of-Distribution (OOD) Generalization
supports
Machine learning generalization when training and test distributions differ; linked to causal invariance.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

If EI maximization is used as a regularization in representation learning, then OOD generalization will improve beyond current invariant risk minimization methods.hypothesis0.837
Proposed conjecture in §4.3.1.
EI maximization serves as an objective standard for selecting coarse-graining and macro-dynamics.claim0.807
Claim by Hoel et al. and endorsed by this survey; used to counter subjectivity critiques.
Out-of-Distribution Probe Generalizationconcept0.782
The capacity of a probe trained on one true/false dataset to accurately classify statements from topically and structurally different datasets
CoT improves in-distribution but may harm out-of-distribution generalizationclaim0.773
Interpretation of scope generalization results
The results generalise readily to non-equilibrium systems where scaling relationships remain similar (e.g., dynamic or localised scaling).claim0.749
Claim about broader applicability of the scaling argument
Representational divergence (as measured by EMD) can predict lower out-of-distribution intervention performanceclaim0.748
Practical utility of reducing divergence demonstrated through regression analysis
Monte Carlo Integration for EImethod0.742
Technique to estimate the continuous EI formula by sampling, used in neural network EI calculation.
Evolutionary Transitions in Individuality (ETI)framework0.735
Theory explaining how new levels of biological organization and individuality emerge through transitions in collective intelligence and problem-solving rescaling.