concept
active
concept:geometry-of-featuresGeometry of features
Research thread within About Blank concerning the structure and relational properties of neural network feature representations; covariance pooling tangentially supports this thread.
Neighborhood — ranked by edge-count
Concepts (2)
concept
- Geometryrelated_toThe actual shapes and spatial relationships of buildings, essential to living structure.
- Second momentsassociated_withStatistical moments capturing pairwise feature co-occurrence patterns; the core insight is that second moments preserve structure that first moments destroy.
Institutes (1)
institute
- About Blankassociated_withResearch group working on AI phenomenology and the Koan Battery.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- General principle supported tangentially by covariance pooling work; relates to feature co-occurrence structure.
- Features identified in Llama-3.1-8B that compute sums using periods respecting base-10 addition (2, 5, 10) rather than concept-specific periods
- Hypothesized extension of superposition where features may be higher-dimensional manifolds rather than 1D directions
- A more complex geometric structure used to characterize in-context learning task representations
- The structured geometry present in the external world (e.g., circles, spatial manifolds) that networks learn to mirror.
- Decoder cosine similarity maps onto concept similarity.
- The spatial/geometric organization of conceptual structure within neural network representations; central to the paper's thesis.
- Domain of techniques for constructing informative features from raw data; covariance pooling is a feature engineering method for token sequences.