claim
active
claim:nla-explanations-can-contain-claims-about-the-target-model-s-input-context-that-are-verifiably-false-but-are-typically-thematically-faithful-to-the-contextNLA explanations can contain claims about the target model's input context that are verifiably false, but are typically thematically faithful to the context.
Key limitation identified: NLAs hallucinate specific details while preserving thematic accuracy; informs practical usage.
Source paper
extracted_fromNeighborhood — ranked by edge-count
Findings (1)
finding
- Illustrates NLA's capture of high-level cognition and hallucination of specifics; corroborated with attribution graphs.
Communities (3)
community
- Spans attention head decomposition, benchmark awareness, and genomic pathogenicity prediction via neural models.
- Using NLAs to extract human-readable explanations of model internals via unsupervised reconstruction, revealing steering vectors, confabulation patterns, and causal belief capture.
- NLA explanations used as steering vectors and auditing tools to investigate model beliefs and misalignment.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- While NLA claims can be false in specifics, they are typically thematically faithful to contextclaim0.879Key insight about confabulation patterns in NLAs enabling practical use.
- Core limitation and usage heuristic: read NLAs for themes rather than individual factual claims; cross-check with original context.
- Can NLAs provide mechanistic grounding of which aspects of an activation drove components of explanations?question0.800Identified as a key limitation: NLAs are blackboxes by construction.
- Downstream task validating NLA utility for model auditing; agents succeed without access to misalignment training data.
- Practical implication.
- A better basis (e.g., factoring tokens into languages or parts of speech) may yield a sparse interpretable modelhypothesis0.791Same model can be dense in one basis and sparse in another.
- Soft clues redundant with transcript.
- A good explanation is one that helps predict model behavior on related counterfactual inputsclaim0.786Central evaluative stance.