EEG Transformer Embeddings

The internal representations of EEG transformers from which SAE features are extracted

Neighborhood — ranked by edge-count

framework

TopK Sparse Autoencoders
implements
The central mechanistic interpretability tool applied across all three EEG transformers to extract sparse feature dictionaries

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

EEG foundation modelsconcept0.745
Large transformer models pretrained on EEG data for clinical tasks; the object of interpretability in this paper.
Are the features extracted by SAEs from EEG transformers monosemantic or entangled?question0.743
Research question motivating the monosemanticity and entanglement benchmarking
Temporal embeddingmethod0.742
Lagged time series used to capture dynamical dependencies.
SAEs successfully extract sparse feature dictionaries from embeddings of SleepFM, REVE, and LaBraM EEG transformers.finding0.741
Foundational empirical result enabling all downstream analysis
Vector Embedding Representationconcept0.731
The specific type of representation studied in the paper: function f: X→R^n assigning feature vectors to inputs
Unsupervised autoencoder embeddingsmethod0.729
Method used alongside covariance pooling for the Gene Ontology prediction task; produces embeddings without large labeled datasets.
Embedding Whiteningconcept0.728
Preprocessing step using dev-set covariance to standardize span embeddings before computing S
Token embeddingsconcept0.725
Vector representations of individual tokens from genomic foundation models; the raw inputs to sequence pooling methods.