claim
active
claim:the-softmax-function-may-be-interpreted-as-performing-lateral-inhibition-leading-to-sharper-inferencesThe softmax function may be interpreted as performing lateral inhibition, leading to sharper inferences.
Interpretation of neural implementation in Section 5.1.
Source paper
extracted_from(2020) · Lancelot Da Costa · Thomas Parr · Noor Sajid · Sebastijan Veselic +2
Neighborhood — ranked by edge-count
Concepts (1)
concept
- Lateral InhibitionsupportsSoftmax interpreted as winner-take-all mechanism that sharpens neural representations.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Neural plausibility argument for softmax policy selection.
- Neuronal dynamics computed from free energy gradients; interpreted as average firing rate of neural populations.
- Using softmax to translate membrane potentials into firing rates, implementing lateral inhibition.
- would place-like representations emerge in memory neurons for activation functions other than softmax?question0.765Open empirical question left for future work about robustness of place cell emergence.
- The approximate posterior over policies is a softmax function of the negative expected free energy.claim0.760Mathematical form of policy selection, eq. (10).
- Superposition hypothesis: neural networks represent more features than dimensions using almost-orthogonal directions.hypothesis0.746Explanation for why dictionary learning can recover many more features than dimensions.
- Key limitation acknowledged by authors.
- Key asymmetry finding interpreted mechanistically by the authors.