Covariance Pooling

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Covariance pooling could generalize beyond genomics as a general-purpose replacement for mean poolingclaim0.833
  Authors' suggestion that the second-moment preservation principle applies broadly, not just to genomic foundation models.
• Covariance pooling preserves joint activation structure (feature co-occurrence) that mean pooling discardsclaim0.820
  Specific interpretive claim about what covariance pooling captures: the pairwise co-activation patterns across features that are invisible to mean pooling.
• Can covariance pooling generalize beyond genomics to other domains?question0.806
  Open question implied by the claim that the method could generalize; empirical validation beyond genomics is not provided in this paper.
• Covariance pooling compresses gigabytes of activations into compact stable embeddings without large labeled datasetsfinding0.790
  Practical finding: the method produces compact fixed-length representations from large volumes of token activations without requiring supervised labels.
• Covariance pooling achieves +52.9% R² improvement over mean pooling on Genomic Track Prediction.finding0.757
  Primary empirical result demonstrating practical utility of covariance pooling method.
• Sequence poolingconcept0.740
  General technique for aggregating variable-length token embedding sequences into fixed-length representations; mean pooling and covariance pooling are instances.
• Organizational Invarianceconcept0.724
• Causal Invarianceconcept0.723
  Property that causal mechanisms remain stable across environments; desirable for OOD.