finding
active
finding:in-a-gemma-2-2b-tfa-2k-dictionary-elements-define-predictive-code-and-8k-novel-code-effective-rank-is-20-30-for-predictive-vs-390-410-for-novel

In a Gemma-2-2B TFA, ~2K dictionary elements define predictive code and ~8K novel code; effective rank is ~20–30 for predictive vs ~390–410 for novel.

Fig. 58.

Source paper

extracted_from
Priors in Time: Missing Inductive Biases for Language Model Interpretability
(2025) · Lubana, Ekdeep Singh · Rager, Can · Hindupur, Sai Sumedh R. · Costa, Valerie +12

Neighborhood — ranked by edge-count

Concepts (1)

concept
  • Alternative sparsity notion proposed for predictive component.