concept
active
concept:fourier-featuresFourier features
Features identified in Llama-3.1-8B that compute sums using periods respecting base-10 addition (2, 5, 10) rather than concept-specific periods
Neighborhood — ranked by edge-count
Papers (1)
paper
Methods (1)
method
- Fourier analysis of neural activationsimplementsMethod used to identify the periodic features and their periods in Llama-3.1-8B's MLP neurons
Concepts (4)
concept
- Ringingassociated_withOscillating off-diagonal similarity from low-dimensional embedding of a curved manifold.
- Generic addition mechanismimplementsThe core finding: Llama-3.1-8B reuses one addition mechanism across all cyclic tasks rather than learning task-specific modular arithmetic
- MLP neuronsimplementsThe sparse set of 28 neurons at layer 18 identified as responsible for Fourier feature computation across all cyclic tasks
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Research thread within About Blank concerning the structure and relational properties of neural network feature representations; covariance pooling tangentially supports this thread.
- Mechanistic claim linking identified Fourier features to base-10 arithmetic
- Interpretable concept representations hypothesized as basic units of computation.
- Variance-fraction test with θ=2 null baseline for modular addition.
- Measure of superposition in toy models.
- Repeated features activating on similar inputs, expected under L1 penalty.
- Fourier features with period 10 contribute to base-10 sum computation in the 28-neuron clusterfinding0.746One of the three base-10 Fourier periods identified in the sparse neuron set
- Method to select d_steer top-activated SAE features for constructing control vectors