claim

active

claim:investigating-the-causal-substructure-of-neural-representations-is-necessary-to-avoid-misidentifying-data-structures-of-simpler-representations-as-abstract-concepts

Investigating the causal substructure of neural representations is necessary to avoid misidentifying data structures of simpler representations as abstract concepts

Motivated by the finding that lexical entailment decomposes into word identities.

Source paper

extracted_from

Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representations

(2023) · Atticus Geiger · Zhengxuan Wu · Christopher Potts · Thomas Icard +1

Neighborhood — ranked by edge-count

Papers (1)

paper

Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representations
introduces

Claims (1)

claim

What appears to be a representation of lexical entailment in BERT is actually a data structure of two word identity representations, not an encoding of the entailment relation
extends
Key asymmetry between hierarchical equality and NLI experiments; BERT stores identities rather than the abstract relation.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Does the geometric structure of neural representations causally shape model behavior?question0.813
The motivating research question of the paper
Neural representations carry rich geometric structure; but does that structure causally shape behavior?quote0.813
Opening sentence framing the paper's core inquiry.
If sub-neural chemical dynamics are crucial for cognition, then neural models ignoring them will fail to capture key cognitive phenomena.hypothesis0.802
Sloman's implicit hypothesis behind his critique of synaptic weight-only models.
Geometric structure of neural representations causally shapes model behaviorclaim0.800
The paper's core causal assertion: geometry is not incidental but mechanistically linked to behavior
In the long run, studying circuit motifs may be more important than studying individual circuits for understanding neural networks.claim0.796
Strategic claim about the relative importance of motif-level abstraction over circuit-level analysis
Smolensky (1986) proposes that viewing a neural representation under a basis that is not aligned with individual neurons can reveal the interpretable distributed structure of the neural representations.quote0.796
Load-bearing theoretical claim providing the conceptual foundation for DAS.
Ultimately, we would like to understand neural networks well enough to be able to intentionally design them.quote0.794
Vision statement in the conclusion.
causal abstraction implicitly relies on strong assumptions about how features are encoded in deep neural networks (DNNs), and becomes trivial without such assumptionsquote0.790
Load-bearing formulation of the paper's central argument