method
active
method:fourier-analysis-of-neural-activationsFourier analysis of neural activations
Method used to identify the periodic features and their periods in Llama-3.1-8B's MLP neurons
Neighborhood — ranked by edge-count
Concepts (1)
concept
- Fourier featuresimplementsFeatures identified in Llama-3.1-8B that compute sums using periods respecting base-10 addition (2, 5, 10) rather than concept-specific periods
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Linear representation hypothesis supported by empirical and theoretical arguments.
- The 28 identified neurons can be partitioned into disjoint clusters each computing a different Fourier period sum
- How important is the geometry of activation space for explaining neural network behavior?question0.753Open question A.1.1a.5.
- Fourier features with period 10 contribute to base-10 sum computation in the 28-neuron clusterfinding0.748One of the three base-10 Fourier periods identified in the sparse neuron set
- Preliminary interpretability of interventions.
- Does the geometric structure of activation space causally shape neural network behavior?question0.742Central research question driving the work.
- Neural Representations of Location Composed of Spatially Periodic Bands (Krupic et al., 2012)concept0.734Discovery of band cells; TEM-t also recapitulates these representations.