High Density Feature Cluster

Mode in feature density histogram around 1e-5 corresponding to interpretable features, contrasted with ultralow density cluster

Neighborhood — ranked by edge-count

concept

Feature Density
associated_with
Fraction of training tokens on which a given feature has nonzero activation; used as proxy metric for autoencoder quality

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

High-Density Housingconcept0.782
Housing at densities of 50–80 families per acre (approximately 200 households per hectare), the design challenge addressed.
Ultralow Density Clusterconcept0.767
Cluster of autoencoder features with extremely low activation density (~1e-7) that are generally not interpretable and appear to be training artifacts
Feature Density Histogrammethod0.764
Log-scale histogram of feature firing rates used as proxy for autoencoder quality during hyperparameter tuning
Complexity Clusterconcept0.756
HDBSCAN Clusteringmethod0.749
Density-based clustering applied to 10-dimensional UMAP to organize feature directions into clusters
The features are often organized in geometrically-related clusters that share a semantic relationship.claim0.743
Decoder cosine similarity maps onto concept similarity.
Customizability Clusterconcept0.743
Cluster 1 (4-cluster): CONTRAST, NOT-SEPARATENESS, ROUGHNESS, ALTERNATING REPETITION, GOOD SHAPEfinding0.733
First cluster of the four-cluster grouping, containing properties 9, 15, 11, 4, 6.