Genomic Foundation Models

Large pretrained models for genomic sequences; subject of this work's pooling improvements.

Neighborhood — ranked by edge-count

paper

framework

Evo-2
associated_with
Genomic foundation model used to predict and explain variant pathogenicity.

method

Covariance Pooling
implements
Novel aggregation technique replacing mean pooling; preserves joint activation structure (feature co-occurrence) in token embeddings.

artifact

Covariance-based Sequence Pooling (2026)
mentions
Goodfire research post introducing covariance pooling as a replacement for mean pooling in genomic foundation models; shows +52.9% R² lift on genomic track prediction.

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Foundation Modelsconcept0.843
Large pretrained models used as backbones across tasks; their universality motivates the convergence hypothesis
Model internals of genomic foundation models can yield mechanistic explanations for variant effectsclaim0.803
Foundational interpretability claim that the paper exemplifies.
EEG foundation modelsconcept0.776
Large transformer models pretrained on EEG data for clinical tasks; the object of interpretability in this paper.
Foundation models trained on different data converge on similar latent representations, suggesting a Platonic form.claim0.744
Generative Modelconcept0.744
Agent's internal probabilistic model of environment; enables belief inference about hidden states given outcomes.
Top-Down Models in Biologyframework0.729
Pezzulo and Levin's framework for explanation and control of complex living systems above the molecular level, cited as prior work by Levin
Mixed Generative Modelsconcept0.728
Generative models combining discrete and continuous states to link decisions and movements.
modelconcept0.725
A representation that captures relevant aspects of a system; according to the theorem, the regulator must embody this.