paper
archived
paper:arithmetic-in-the-wild-llama-uses-base-10-addition-to-reason-about-cyclic-conceptsArithmetic in the Wild: Llama uses Base-10 Addition to Reason About Cyclic Concepts
BySheridan Feucht·Tal Haklay·Usha Bhalla·Daniel Wurgaft·Can Rager·Raphaël Sarfatiⓘ+6 moreGoodfire, Harvard University + 2 more
Neural Steering MethodsBase-10 additionCausal abstraction analysisCausal abstractionCyclic concept reasoning probingCyclic ConceptsFourier analysis of neural activationsDisjoint neuron clusters for Fourier periodsNeuron cluster identification via partitioningGeneric addition mechanismGeometry of featuresLlama-3.1 8BMechanistic InterpretabilityTwo-stage cyclic computation
Methods (4)
- Causal abstraction analysisThe formal method used to establish that the identified circuit causally mediates the model's cyclic reasoning behavior
- Cyclic concept reasoning probingExperimental paradigm using prompts like 'what month is six months after August?' to study model arithmetic
- Fourier analysis of neural activationsMethod used to identify the periodic features and their periods in Llama-3.1-8B's MLP neurons
- Neuron cluster identification via partitioningMethod used to identify and partition the 28 MLP neurons into disjoint clusters by Fourier period