claim
active
claim:automated-interpretability-struggles-with-influential-features-on-special-tokens-and-punctuation-labeling-them-as-the-tokens-rather-than-the-context-informationAutomated interpretability struggles with influential features on special tokens and punctuation, labeling them as the tokens rather than the context information
Known limitation.
Source paper
extracted_fromNeighborhood — ranked by edge-count
Papers (1)
paper
Methods (1)
method
- Claude-based explanation of features from max-activating examples.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Method using large language models (Claude) to generate and test explanations of features at scale
- Design choice validated empirically.
- The capability to explain model predictions; a central theme of the paper, with disruption profiles as vehicle.
- Motivates shift from studying model activations ('thoughts') to understanding parameters ('the computations themselves').
- A better basis (e.g., factoring tokens into languages or parts of speech) may yield a sparse interpretable modelhypothesis0.778Same model can be dense in one basis and sparse in another.
- New affordance of features beyond monitoring/steering.
- Benchmark discipline.