paper
referenced-only
2021
paper:geva-transformer-feed-forward-layers-are-key-2021Transformer feed-forward layers are key-value memories
ByMor Geva·Roei Schuster·Jonathan Berant·Omer Levy
Related work— refs + corpus + external arXiv
Cited / in-corpus / arXiv badges show which signals surfaced each row. Multi-source rows weighted higher.
- Diversity of Transformer Layers: One Aspect of Parameter Scaling LawsYing Zhang, Jingun Kwon, Katsuhiko Hayashi, Manabu Okumura, Taro Watanabe Hidetaka Kamigaito2025≈ 74%
- Fighter: Unveiling the Graph Convolutional Nature of Transformers in Time Series ModelingWeixin Bu, Wendong Xu, Runsheng Yu, Yik-Chung Wu, Ngai Wong Chen Zhang2025≈ 72%
- Transformer Dynamics: A neuroscientific approach to interpretability of large language modelsJesseba Fernando and Grigori Guitchounts2025≈ 72%
- ≈ 71%
- Birth of a Transformer: A Memory ViewpointVivien Cabannes, Diane Bouchacourt, Herve Jegou, Leon Bottou Alberto Bietti2023≈ 71%
- ≈ 71%
- ≈ 70%
- The Parallelism Tradeoff: Limitations of Log-Precision TransformersWilliam Merrill and Ashish Sabharwal2023≈ 70%
- A Mathematical Framework for Transformer Circuitsin corpus2021≈ 70%
- Transformer Circuit Faithfulness Metrics are not RobustBilal Chughtai, William Saunders Joseph Miller2024≈ 69%
- Circuit Transformer: A Transformer That Preserves Logical EquivalenceXing Li, Lei Chen, Xing Zhang, Mingxuan Yuan, Jun Wang Xihan Li2025≈ 69%
- ≈ 69%
- Beyond Components: Singular Vector-Based Interpretability of Transformer CircuitsAreeb Ahmad and Abhinav Joshi and Ashutosh Modi2025≈ 69%
- Explaining the Explainer: Understanding the Inner Workings of Transformer-based Symbolic Regression ModelsArco van Breda and Erman Acar2026≈ 69%
- Inner Loop Inference for Pretrained Transformers: Unlocking Latent Capabilities Without TrainingVincent Gripon, Bastien Pasdeloup, Axel Marmoret, Lukas Mauch, Fabien Cardinaux, Ghouthi Boukli Hacene Jonathan Lys2026≈ 69%
- ≈ 69%
- What One Cannot, Two Can: Two-Layer Transformers Provably Represent Induction Heads on Any-Order Markov ChainsMarco Bondaschi, Nived Rajaraman, Jason D. Lee, Michael Gastpar, Ashok Vardhan Makkuva, Paul Pu Liang Chanakya Ekbote2025≈ 69%
- Relating transformers to models and neural representations of the hippocampal formationin corpus2021≈ 68%
- Zoom In: An Introduction to Circuitsin corpus2020≈ 66%
- Self-Improvising Memory: A Perspective on Memories as Agential, Dynamically Reinterpreting Cognitive Gluein corpus2024≈ 65%
- The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetsin corpus2023≈ 63%
- Learning without neurons in physical systemsin corpus2022≈ 63%
- ≈ 63%
- ≈ 62%
- The Non-Linear Representation Dilemma: Is Causal Abstraction Enough for Mechanistic Interpretability?in corpus2025≈ 62%
- ≈ 62%
- ≈ 62%
- ≈ 62%
- ≈ 62%
- ≈ 62%
Similar preprints — Semantic Scholar
Cited by (2)
- Steering at the Source: Style Modulation Heads for Robust Persona Control
Residual-stream activation steering reliably degrades text coherency when steering vectors push models toward out-of-distribution behavior, and this collapse goes undetected by standard benchmarks: MM
- Arithmetic in the Wild: Llama uses Base-10 Addition to Reason About Cyclic Concepts
Llama-3.1-8B solves cyclic arithmetic (e.g., "what month is six months after August?") not by performing modular addition in the period of the cyclic concept (12 for months, 7 for days of the week) as