claim

active

claim:the-feed-forward-network-truly-implements-a-symbolic-tree-structured-algorithm-for-hierarchical-equality-with-abstract-equality-relations-not-decomposable-into-input-identities

The feed-forward network truly implements a symbolic, tree-structured algorithm for hierarchical equality, with abstract equality relations not decomposable into input identities

DAS reveals that the neural network encodes abstract relational structure rather than raw input identities.

Source paper

extracted_from

Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representations

(2023) · Atticus Geiger · Zhengxuan Wu · Christopher Potts · Thomas Icard +1

Neighborhood — ranked by edge-count

Papers (1)

paper

Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representations
introduces

Findings (2)

finding

Identity Subspace of Left Equality model achieves ~0.50 IIA, indicating equality relations cannot be decomposed into input identities
associated_withsupports
DAS reveals that the network encodes abstract equality relations rather than storing identities of inputs.
DAS achieves 100% IIA on hierarchical equality task with |N|=16, intervention size 8, Layer 1
supports
DAS discovers a perfect alignment between the feed-forward network and the Both Equality Relations high-level model.

Claims (1)

claim

What appears to be a representation of lexical entailment in BERT is actually a data structure of two word identity representations, not an encoding of the entailment relation
contradicts
Key asymmetry between hierarchical equality and NLI experiments; BERT stores identities rather than the abstract relation.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Does the hierarchical equality network implement a program that computes w=x and y=z as intermediate values?question0.799
Specific research question for the first experiment.
Hierarchical clique-structured networks can maintain multiscale coherence; biology evolved hierarchy and stigmergy as coherence solutions.claim0.766
Theorem 1: Any algorithm is an input-restricted distributed abstraction of any DNN satisfying mild assumptionsfinding0.762
Central theoretical result proving unrestricted causal abstraction is trivial
Representing non-linearly separable functions requires a network with multiple layers.claim0.760
Architectural requirement from machine learning.
The discovery of perfect abstract equality representations that cannot be decomposed into entity representations is a foundational result informing our understanding of how symbolic and connectionist architectures coexistclaim0.753
Concluding claim about theoretical significance of the hierarchical equality finding.
Can an interpretable symbolic algorithm be used to faithfully explain a complex neural network model?question0.750
Framing question for the paper's research program.
Friedman associates the non-hierarchical communication network to the participatory dimension of a democratic society.claim0.748
Interpretation that Friedman’s use of Baran’s distributed networks is a political statement for a non-paternalistic society.
Hierarchical structure in interaction topology enables complex multiscale patterns that cannot exist in flat networks.claim0.747
Explains why biological systems achieve organization across scales while language models struggle; grounds in free energy scaling