question
active
question:how-is-semantic-meaning-encoded-in-activationsHow is semantic meaning encoded in activations?
Mechanistic question AOs can't answer.
Source paper
extracted_from(2025) · Adam Karvonen
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Defines scope; they cannot answer encoding, geometry, or algorithm questions.
Questions (1)
question
- Are concepts encoded linearly?associated_withMechanistic open question.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Internal representations of the model on which probes operate; the method uses activations to rank datapoints.
- Assumption that structured inputs act as anchors biasing activation toward target patterns PT.
- Explanation for why probes on h_b achieve perfect accuracy but h_s probes succeed only 0.60% of the time
- Motivates shift from studying model activations ('thoughts') to understanding parameters ('the computations themselves').
- Meaning that arises from relations within the graphical system, not inherent in elements.
- The central idea that external structure binds latent patterns to desired targets.
- Secondary question motivating the IIT analysis; asks whether LLM hidden states contain something beyond propositional content.
- Argument for scalability.