question
active
question:does-the-hierarchical-equality-network-implement-a-program-that-computes-w-x-and-y-z-as-intermediate-valuesDoes the hierarchical equality network implement a program that computes w=x and y=z as intermediate values?
Specific research question for the first experiment.
Source paper
extracted_from(2023) · Atticus Geiger · Zhengxuan Wu · Christopher Potts · Thomas Icard +1
Neighborhood — ranked by edge-count
Findings (1)
finding
- DAS achieves 100% IIA on hierarchical equality task with |N|=16, intervention size 8, Layer 1answered_byDAS discovers a perfect alignment between the feed-forward network and the Both Equality Relations high-level model.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- DAS reveals that the neural network encodes abstract relational structure rather than raw input identities.
- Replicates Geiger et al. 2024b pattern of layer-dependent IIA degradation with linear maps
- Task where the input is a sequence w,x,y,z and the label is (w=x)=(y=z); used to test relational reasoning in developmental/cognitive psychology.
- Brute-force search achieves best IIA of 0.60 on hierarchical equality Both Equality Relations in Layer 1finding0.730DAS substantially outperforms brute-force search (1.00 vs 0.60 IIA) on the hierarchical equality task.
- Explains why biological systems achieve organization across scales while language models struggle; grounds in free energy scaling
- Algorithm computing both equality relations separately before comparing them in hierarchical equality task
- Exception to the general trend; attributed to insufficient RevNet capacity rather than algorithm not being implemented