finding

active

finding:lexical-entailment-representation-decomposes-into-word-identity-sub-representations-with-0-97-0-98-iia-lexeme-subspace-of-lexical-entailment

Lexical entailment representation decomposes into word identity sub-representations with ~0.97-0.98 IIA (Lexeme Subspace of Lexical Entailment)

In contrast to hierarchical equality, lexical entailment in BERT decomposes into representations of word identities, not a single abstract relation.

Source paper

extracted_from

Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representations

(2023) · Atticus Geiger · Zhengxuan Wu · Christopher Potts · Thomas Icard +1

Neighborhood — ranked by edge-count

Claims (1)

claim

What appears to be a representation of lexical entailment in BERT is actually a data structure of two word identity representations, not an encoding of the entailment relation
associated_withsupports
Key asymmetry between hierarchical equality and NLI experiments; BERT stores identities rather than the abstract relation.

Questions (1)

question

Can the distributed representation of lexical entailment be decomposed into representations of the individual word identities?
answered_by
Research question leading to the key NLI finding about word identity data structures.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

DAS achieves 100% IIA for combined Negation and Lexical Entailment model on MoNLI at Layer 9, intervention size 256finding0.778
Perfect abstraction relation between BERT and symbolic algorithm with negation and lexical entailment variables.
Identity Subspace of Left Equality model achieves ~0.50 IIA, indicating equality relations cannot be decomposed into input identitiesfinding0.750
DAS reveals that the network encodes abstract equality relations rather than storing identities of inputs.
NLAs bridge unsupervised concept-discovery methods (e.g., SAEs) and supervised activation-verbalization methods (e.g., activation oracles)claim0.748
The paper positions NLAs as combining unsupervised learning with direct readability.
Lexical Entailmentconcept0.746
The semantic relation between words wp and wh (entails/neutral) used as an intermediate variable in the MoNLI high-level model.
The discovery of perfect abstract equality representations that cannot be decomposed into entity representations is a foundational result informing our understanding of how symbolic and connectionist architectures coexistclaim0.746
Concluding claim about theoretical significance of the hierarchical equality finding.
Each expression denotes something, depending only on denotations of subexpressions.claim0.744
Property of denotative programming from Landin.
It's tricky, because for a typical language model the entity is sort of tricameral: the base simulator, the simulated simulator, and the simulated awareness.quote0.740
Antra's earlier definitive statement of the tricameral model.
Towards Monosemanticity: Decomposing Language Models with Dictionary Learning (Bricken et al., 2023)concept0.739
Foundational SAE mechanistic interpretability paper