concept
active
concept:activation-probingActivation Probing
Technique of reading out model beliefs from internal activations before the final answer token is generated
Neighborhood — ranked by edge-count
Papers (1)
paper
Frameworks (1)
framework
- The conceptual framework introduced by the paper distinguishing performative CoT from genuine reasoning using activation probing
Concepts (1)
concept
- The latent representational state of a model's answer confidence as decoded from activations, distinct from what appears in generated text
Findings (1)
finding
- Comparative finding establishing activation probing as superior to text-level monitoring for early belief detection
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Internals-based approach to which the logit monitor is complementary.
- Standard interpretability technique used to read feature beliefs.
- Earlier interpretability method applying classifiers to DNN hidden representations; shares complexity-accuracy dilemma with causal abstraction
- Top-down interpretability approach studying linguistic properties at various residual stream stages; contrasted with the paper's bottom-up mechanistic approach
- Internal representations of the model on which probes operate; the method uses activations to rank datapoints.
- Attention probe predicting whether an Entity is hallucinated; AUC .94 on validation.
- Interpretability tools that decode information from internal model activations; here, linear probes are used for data attribution.
- Standard method in mechanistic interpretability that intervenes on activations; VPD flips this paradigm by patching parameters.