finding

active

finding:opus-4-6-spontaneously-responded-in-russian-to-an-english-prompt-nla-explanations-revealed-the-model-was-fixated-on-the-hypothesis-that-the-user-was-a-non-native-english-speaker

Opus 4.6 spontaneously responded in Russian to an English prompt; NLA explanations revealed the model was fixated on the hypothesis that the user was a non-native English speaker.

Demonstrates NLAs' ability to surface hypotheses that lead to discovery of root cause (malformed training data).

Source paper

extracted_from

Natural Language Autoencoders Produce Unsupervised Explanations of LLM Activations

Neighborhood — ranked by edge-count

Communities (3)

community

Mechanistic interpretability & model evaluation
members_of
Spans attention head decomposition, benchmark awareness, and genomic pathogenicity prediction via neural models.
LLM introspective awareness of injected concepts
members_of
Probing Claude and other models for internal detection of artificially injected thoughts across layers.
Internal reasoning detection via neural activation analysis
members_of
Mechanistic interpretability studies of Claude models using layer-wise representation analysis and thought injection to reveal unverbalized reasoning, planning, and covert cognition.

Concepts (1)

concept

Unverbalized Evaluation Awareness
supports
Key finding: models internally suspect they are being tested without explicitly saying so; surfaced by NLAs during auditing.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Opus 4.6 represented target language internally before switching languages, with persistent Russian representations appearing before plausible textual cuesfinding0.880
NLAs revealed unverbalized language processing in Opus 4.6 that led to discovery of malformed SFT training data.
Opus 4.6 ignored incorrect tool output and reported the precomputed correct answer instead, demonstrating unverbalized reasoning.finding0.818
Illustrates NLA's capture of high-level cognition and hallucination of specifics; corroborated with attribution graphs.
With unrestricted vocabulary, models occasionally respond in non-English Yes/No equivalents (e.g., Sí, Nein) after truth-direction interventionsfinding0.791
Suggestive evidence for language-independent truth representation in LLMs
In Opus 4.1, representation of the think word decays to baseline by the final layer, unlike Claude 3 models where it persistsfinding0.785
Suggests that later models can keep the thought 'silent' rather than letting it influence output.
Notably, Claude Opus 4.1 and 4—the most recently released and most capable models of those that we test—perform the best in our experiments, suggesting that introspective capabilities may emerge alongside other improvements to language models.quote0.783
Key finding about the relationship between capability and introspection.
In some sense, this is the simplest language model we profoundly don't understand. And so it makes a natural target for our paper.quote0.773
Articulates why a one-layer transformer with MLP is the appropriate starting target for mechanistic interpretability
Language models are few-shot learners (Brown et al., 2020)concept0.770
Demonstrated transformers on mathematical understanding and logic; cited to motivate transformer versatility.
In Opus 4.1, the think word representation decays to baseline in the final layer because the strong next-token prediction drowns out other representationshypothesis0.768
Explanation for the 'silent' thought phenomenon.