SyntaxGym

Online platform for targeted evaluation of language models that CausalGym adapts

Neighborhood — ranked by edge-count

framework

CausalGym
extends
Multi-task benchmark of linguistic behaviours for measuring causal efficacy of interpretability methods, adapted from SyntaxGym

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Shape Grammarsframework0.732
A computational design method based on shape rewriting rules, developed by George Stiny; noted as academic and not yet practically useful for living structure.
Shape Grammarmethod0.730
Grammar as Positional Encoding for Languagehypothesis0.712
Hypothesis that in language tasks, the abstract structure encoded in positional encodings corresponds to grammatical structure.
Stigmergyconcept0.709
Coordination mechanism where subunits leave messages in an external medium; stress sharing functions similarly via leaked stress molecules.
CausalGym covers only linguistic tasks; benchmarking interpretability methods on non-linguistic behaviours remains openquestion0.702
Identified limitation calling for broader task coverage in future work
Language Modelsconcept0.701
Primary substrate for manifold steering experiments; demonstrates method on reasoning and in-context tasks.
Language Modelconcept0.697
Primary test domain for manifold steering, including reasoning and ICL tasks
Turing Machineconcept0.696
Theoretical construct establishing classical demarcation between machine and environment via input/output channels.