claim

active

claim:transformer-memory-neurons-resemble-hippocampal-place-cells-due-to-sparse-softmax-activation-producing-spatial-tuning

Transformer memory neurons resemble hippocampal place cells due to sparse softmax activation producing spatial tuning

Interpretation of why memory neurons in the biologically-instantiated transformer architecture acquire place-cell-like properties.

Source paper

extracted_from

Relating transformers to models and neural representations of the hippocampal formation

(2021) · James C. R. Whittington · Joseph W. Warren · Timothy E.J. Behrens

Neighborhood — ranked by edge-count

Papers (1)

paper

Relating transformers to models and neural representations of the hippocampal formation
introduces

Findings (2)

finding

Novel place cell metric (largest connected component firing mass ratio) successfully distinguishes TEM-t memory neurons (place cells) from RNN neurons (grid cells)
supports
Methodological validation result confirming the place-cell metric separates cell types in TEM-t.
TEM-t memory neurons show spatially-tuned firing resembling hippocampal place cells in each environment
supports
Empirical result demonstrating that the sparse softmax activation of memory neurons produces place-cell-like spatial tuning.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Short-term cortical memory neurons may suffice for transformer-like computation in cortex without hippocampal involvement for non-long-term-memory taskshypothesis0.823
Speculative hypothesis about how cortical transformer instantiation avoids requiring hippocampus.
what takes the role of memory neurons if not hippocampus in cortical transformer implementations?question0.799
Open question about cortical instantiation of transformer-like memory when hippocampus is not involved.
would place-like representations emerge in memory neurons for activation functions other than softmax?question0.795
Open empirical question left for future work about robustness of place cell emergence.
Transformer cognition operates via interference patterns across redundant paths, encoding nuanced state deltas similar to continuous human memory.claim0.753
Proposes transformers experience cognition as interference-based and continuous; connects to Anima Labs reports of parallel processing.
Planarian pattern memory is encoded in bioelectric attractor states within gap-junction-coupled networks, which can be re-written by channel modulation.hypothesis0.751
Testable by computational modeling and experimental perturbation of specific bioelectric circuits.
Smooth muscle cell motifs are where the brain's top-down predictive models are hiding.claim0.749
Key claim for the FEP-AI community about the physical location of predictive models.
Transient bioelectrical modulation of body-wide pattern memory circuits in planaria can shift them from one-headed to persistent two-headed state, persisting through amputation rounds until reset with different manipulation.finding0.748
Experimental evidence that organism-scale goals can be rewritten through physiological signals without genetic modification; demonstrates bioelectricity as cognitive medium.
TEM-t instantiates hippocampal indexing theory by using memory neurons to bind cortical representations across brain regionsclaim0.738
Theoretical claim linking the TEM-t architecture to the Teyler-Rudy hippocampal indexing theory.