claim
active
claim:activation-oracles-are-a-fundamentally-non-mechanistic-techniqueActivation Oracles are a fundamentally non-mechanistic technique
Defines scope; they cannot answer encoding, geometry, or algorithm questions.
Source paper
extracted_from(2025) · Adam Karvonen
Neighborhood — ranked by edge-count
Papers (1)
paper
Questions (2)
question
- Mechanistic question AOs can't answer.
- Mechanistic question AOs can't answer.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Framework training LLMs to answer questions about externally-provided activation vectors
- Supervised method training models to answer questions about activations; NLAs differ by being unsupervised.
- Central claim from tl;dr.
- The paper positions NLAs as combining unsupervised learning with direct readability.
- Core framing of AOs.
- Internal representations of the model on which probes operate; the method uses activations to rank datapoints.
- Motivates shift from studying model activations ('thoughts') to understanding parameters ('the computations themselves').
- Activations may be inherently incomplete for causal claims, requiring weight- or circuit-level accesshypothesis0.729One explanation for tool failure.