claim

active

claim:there-appears-to-be-a-systematic-relationship-between-the-frequency-of-concepts-and-the-dictionary-size-needed-to-resolve-features-for-them

There appears to be a systematic relationship between the frequency of concepts and the dictionary size needed to resolve features for them.

Feature presence depends on concept frequency in training data, with a threshold scaling inversely with alive features.

Source paper

extracted_from

Scaling monosemanticity: Ex-tracting interpretable features from claude 3 sonnet

Neighborhood — ranked by edge-count

Findings (1)

finding

The likelihood of a dedicated feature for a concept (element, city, animal, food) follows a sigmoid in log-frequency of the concept in training data, with threshold frequency inversely proportional to number of alive features.
supports
Quantitative relationship between concept frequency and feature presence.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

The features are often organized in geometrically-related clusters that share a semantic relationship.claim0.801
Decoder cosine similarity maps onto concept similarity.
For a given task, the number of all sequences which work is tiny by comparison with the huge number of all possible sequences; less than a trillionth of all 6 × 10^23 possible sequences actually work well enough.claim0.798
A combinatorial argument that good sequences are astronomically rare, emphasizing the difficulty of discovery.
The examples of features found in language models suggest they are highly sparse variables, consistent with dictionary learning being applicablehypothesis0.787
Motivation for using sparsity-based dictionary learning on language models
what is the 'correct number of features' for dictionary learning, and is this question well-posed?question0.784
Open question about whether there is a true discrete feature count or a continuous splitting process
What Is The Relationship Between The Size Ofquestion0.783
The degree of similarities which exist in a structure must correspond exactly to the degree of similarity of the conditions there, and the degrees of differences which exist in a structure must also correspond to the degrees of difference in the conditions there.claim0.780
The profound principle that underlies all living structure; symmetry as the mathematical trace of necessity.
what metrics can reliably tell us if dictionary learning has successfully extracted high quality features?question0.780
Central methodological gap: current metrics (loss, density histograms, manual inspection) are inadequate
There is a bidirectional relationship between the geometry of representation and behavior across tasks and modalities.claim0.779
Author’s interpretive claim that the shared geometry is general and robust.