Atticus Geiger

Authored papers (7)

• Arithmetic in the Wild: Llama uses Base-10 Addition to Reason About Cyclic Concepts2026

  Llama-3.1-8B solves cyclic arithmetic (e.g., "what month is six months after August?") not by performing modular addition in the period of the cyclic concept (12 for months, 7 for days of the week) as its circularly structured representations might suggest, but by reusing a single generic base-10 addition mechanism across all such tasks. The mechanism operates in two stages: first, the model computes the raw integer sum of its inputs (six + August = 14), then maps that sum back into cyclic concept space (14 → February). The paper introduces a combined causal-abstraction and feature-geometry analysis to demonstrate that Llama-3.1-8B uses task-agnostic Fourier features with periods characteristic of base-10 arithmetic—specifically periods 2, 5, and 10—rather than the concept-native period. Strikingly, a sparse set of only 28 MLP neurons at layer 18 (approximately 0.2% of that layer's MLP) is responsible for this computation across all cyclic tasks and can be partitioned into disjoint clusters, each computing the sum for a Fourier feature of a distinct period. This implies that structured representational geometry does not entail correspondingly structured computation: a model can build domain-specific circular embeddings and yet delegate the actual arithmetic to a domain-general, base-10 calculator reused opportunistically, suggesting that mechanistic interpretability analyses that read off algorithms from representation geometry alone will systematically mislead.

• Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behavior2026

  Manifold steering — intervening on model activations along paths constrained to lie on a learned activation manifold M_h rather than along Euclidean linear directions — produces behavioral trajectories that track the corresponding behavior manifold M_y, while linear (Euclidean) steering cuts through off-manifold regions and generates unnatural outputs. The paper fits M_h to internal representations and M_y to output probability distributions, then tests their bidirectional correspondence via controlled interventions across language models and a video world model. In language models, tasks with cyclic geometries, sequential geometries, and complex graph geometries (in-context learning) all show that manifold-constrained interventions keep behavioral outputs on M_y, whereas linear steering deviates measurably. In a video world model, interventions shaped by physical-dynamics geometry similarly respect M_y. Crucially, the relationship is bidirectional: optimizing interventions in activation space to produce paths along M_y recovers activation trajectories that trace the curvature of M_h. This implies that the core problem of mechanistic steering should be recast not as finding the right direction in a flat Euclidean activation space, but as identifying the correct geometric structure — because representational geometry is not incidental to model behavior but is causally constitutive of it.

• Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behavior2026

  merged

  Manifold steering — intervening along activation-space paths defined by the intrinsic geometry of a fitted activation manifold M_h rather than along Euclidean linear directions — produces behavioral trajectories that remain on the behavior manifold M_y, whereas standard linear steering cuts through off-manifold regions and generates unnatural outputs. Across language model reasoning tasks with cyclic, sequential, and graph geometries, and in a video world model whose task geometry corresponds to physical dynamics, the bidirectional relationship M_h ↔ M_y holds: steering that respects M_h tracks M_y, and optimizing interventions in activation space to follow M_y recovers activation trajectories that trace the curvature of M_h. The method fits two manifolds — one to intermediate representations, one to output probability distributions — and then uses geodesic-like paths on M_h as the intervention substrate rather than a single steering vector. This work argues that neural representation geometry is not incidental but is the causal structure organizing model behavior, which implies that the core problem of model steering should be reframed from finding the right direction to finding the right geometry.

• Reasoning Theater: Disentangling Model Beliefs from Chain-of-Thought2026

  Reasoning models generate chains of thought that are frequently performative rather than causally necessary for reaching the correct answer: on MMLU recall questions, activation probes decode the model's final answer from internal representations far earlier in the chain-of-thought than a CoT language monitor can, demonstrating that the visible reasoning tokens are generated after the belief is already settled. Tested across DeepSeek-R1 671B and GPT-OSS 120B using three complementary methods — activation probing, early forced answering, and a CoT monitor — the analysis reveals a task-difficulty gradient: the gap between probe-decodable certainty and monitor-detectable certainty is large for easy MMLU items but narrows substantially for difficult GPQA-Diamond multihop questions, where genuine step-by-step uncertainty persists longer. The paper introduces probe-guided early exit, which terminates generation once probe confidence passes a threshold, cutting token counts by up to 80% on MMLU and 30% on GPQA-Diamond while preserving accuracy. A key finding that complicates a purely dismissive view is that inflection-point behaviors — backtracking and 'aha' moments — appear almost exclusively in traces where probes register large belief shifts, meaning these surface signals do track genuine uncertainty rather than being uniformly theatrical. The paper argues that this dissociation between internal belief and continued CoT generation is systematic and exploitable, positioning attention probing as both a diagnostic instrument for faithfulness research and a practical mechanism for adaptive computation.

• pyvene: A Library for Understanding and Improving PyTorch Models via Interventions2024ⓒ 1

  pyvene is an open-source Python library that unifies intervention-based research on PyTorch neural models by treating the intervention itself—rather than model surgery code—as the primitive abstraction, expressed in a serializable dict-based configuration that can be shared via HuggingFace. Prior libraries (BauKit, TransformerLens, nnsight, graphpatch, Transformer Debugger) either lack extensibility to recurrent and convolutional architectures or require sophisticated custom code for multi-source, cross-forward-pass interventions; pyvene resolves both limitations with Getter/Setter hooks that track state variables enabling intervention at arbitrary time steps in GRU and other recurrent models. The library ships with trainable intervention types including RotatedSpaceIntervention (Distributed Alignment Search), LowRankRotatedSpaceIntervention, and BoundlessRotatedSpaceIntervention, and reproduces Meng et al.'s factual-association localization result in GPT2-XL in approximately 20 lines of code. A second case study on Pythia-6.9B demonstrates that a 1D DAS intervention finds sparse, causally localized gender representations across layers, whereas a linear probe achieves near-100% classification accuracy almost everywhere—implying that high probe accuracy is insufficient evidence of causal relevance, and that trainable interventions provide a strictly more diagnostic test of whether a representation is mechanistically load-bearing for a behavior.

• Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representations2023ⓒ 9

  Distributed alignment search (DAS) resolves two blocking limitations of prior causal abstraction work—brute-force alignment search and the localist assumption that high-level variables map to disjoint neuron sets—by using gradient descent over orthogonal rotation matrices to find alignments in non-standard bases of neural representations. On a hierarchical equality task, a three-layer feed-forward network with hidden size 16 achieves 100% interchange intervention accuracy (IIA) under DAS at layer 1 with an 8-dimensional intervention subspace, whereas the best brute-force localist search reaches only 0.60 IIA and the closest localist alignment only 0.73 IIA. On the Monotonicity NLI benchmark, BERT-base fine-tuned on MoNLI achieves 100% IIA at layer 9 when 256 non-standard basis dimensions of the [CLS] token encode lexical entailment and 256 others encode negation, while no localist alignment exceeds 0.51 IIA on the same task. A subsequent subspace decomposition reveals a structural asymmetry: the hierarchical equality representations of w=x and y=z cannot be decomposed into representations of individual input identities (subspace DAS IIA ≈ 0.50–0.51), whereas the apparent lexical-entailment representation in BERT decomposes almost perfectly (IIA ≈ 0.97–0.98) into two word-identity representations. DAS implies that previous negative or weak causal abstraction findings may have been artifacts of the localist assumption, and that neural networks can genuinely implement tree-structured symbolic algorithms—but that apparent relational representations may sometimes be data structures over entity identities rather than true relational encodings.

• Arithmetic in the Wild: Llama uses Base-10 Addition to Reason About Cyclic Concepts

  archived

More papers — OpenAlex / S2

Their work is cited by (7)

• CausalGym: Benchmarking causal interpretability methods on linguistic tasks6× refs
• The Non-Linear Representation Dilemma: Is Causal Abstraction Enough for Mechanistic Interpretability?6× refs
• Addressing divergent representations from causal interventions on neural networks6× refs
• Model Alignment Search3× refs
• pyvene: A Library for Understanding and Improving PyTorch Models via Interventions3× refs
• Arithmetic in the Wild: Llama uses Base-10 Addition to Reason About Cyclic Concepts3× refs
• Persona Non Grata: Single-Method Safety Evaluation Is Incomplete for Persona-Imbued LLMs2× refs

Recent mentions (7)

• papers-typed

  feucht-2026-arithmetic.md
• papers-typed

  boppana-2026-reasoning.md
• papers-typed

  boppana-goodfire-reasoning-theater-2026.md
• papers-typed

  paper.md
• papers-typed

  wurgaft-2026-manifold.md
• papers-typed

  feucht-goodfire-geometric-calculator-2026.md
• papers-typed

  geiger-goodfire-world-inside-neural-networks-2026.md