Kernel Density Estimation (KDE)

Used in NIS+ to estimate natural distribution p(yt) for inverse probability weight.

Neighborhood — ranked by edge-count

framework

Neural Information Squeezer Plus (NIS+)
implements
Extension of NIS that directly maximizes effective information using probability reweighting.

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

KDE Density Scoremethod0.828
Nonparametric density estimate scoring how typical an intervened representation is relative to the natural distribution
Feature Densityconcept0.730
Fraction of training tokens on which a given feature has nonzero activation; used as proxy metric for autoencoder quality
Pointwise Mutual Information Kernelconcept0.697
The kernel that contrastive learners converge to; similarity equals PMI between observations
Feature Density Histogrammethod0.692
Log-scale histogram of feature firing rates used as proxy for autoencoder quality during hyperparameter tuning
conditional densitiesconcept0.687
Time-dependent densities of states and parameters obtained by maximizing free energy.
Kernel (representational)concept0.683
A function characterizing how a representation measures distance/similarity between datapoints; used to compare representations
KL Divergenceconcept0.682
A measure of the difference between two probability distributions, used extensively in free energy formulations.
High Density Feature Clusterconcept0.680
Mode in feature density histogram around 1e-5 corresponding to interpretable features, contrasted with ultralow density cluster