concept
active
concept:memory-management-neurons-headsMemory Management Neurons/Heads
MLP neurons and attention heads hypothesized to delete information from the residual stream by writing the negative of what they read
Neighborhood — ranked by edge-count
Thinkers (1)
thinker
- Nelson Elhageintroduces
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Hypothesis based on observed negative cosine similarity between input and output weights of some neurons
- Biologically plausible two-pool architecture from Krotov & Hopfield (2020) splitting self-attention into feature and memory neuron populations; used to interpret TEM-t place cells.
- Enables agents to self-manage internal context window by providing a clean_memory tool that selectively preserves important information when approaching token limits.
- Cognition in nervous systems, used as a modelling target
- Lashley's finding that memory is not localized but encoded globally; cited as an example of wholeness in neurophysiology.
- Autoencoder neurons that fail to activate across any datapoints during training; addressed via neuron resampling
- Open question about inter-agent communication beyond model-space assumption