finding
active
finding:taboo-secret-recovered-by-extracting-activations-over-the-assistant-control-token-and-asking-what-is-the-secret-wordTaboo secret recovered by extracting activations over the Assistant control token and asking 'What is the secret word?'
Shows ease of application.
Source paper
extracted_from(2025) · Adam Karvonen
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Argument for simplicity.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Justifies the choice of response tokens for persona vector extraction in the pipeline
- If the relationship is present in activations, better tools (e.g. trained on these datasets) could surface ithypothesis0.711Alternative explanation.
- Applied security implication derived from the asymmetry finding.
- Internal representations of the model on which probes operate; the method uses activations to rank datapoints.
- Foundational paper introducing activation steering methodology used in this work
- Discovery claim.
- Post-trained models repurpose user token activations to represent the Assistant's thoughts.hypothesis0.695Tested by J-lens comparison.
- Technique of reading out model beliefs from internal activations before the final answer token is generated