Control task for causal evaluation

Adaptation of Hewitt and Liang control tasks to CausalGym: next-token labels replaced with arbitrary tokens to measure method expressivity

Neighborhood — ranked by edge-count

paper

method

Selectivity
uses
Adapted control task metric measuring difference between odds-ratio on original task and arbitrary-label control task

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Intentional control taskmethod0.834
Task instructing the model to write a sentence while thinking or not thinking about a word, measuring internal representation strength.
Causal powerconcept0.789
The ability of an agent to be a driver of subsequent events; a hallmark of cognition that causal emergence quantifies.
controlconcept0.777
The act of directing a system's behavior; the objective of a regulator.
Causal Abstraction over Cyclic Tasksconcept0.776
Causal Structural Probemethod0.774
Probe method combining causal interventions and structural analysis, supported by pyvene's activation collection
Causal abstractionconcept0.773
A framework the paper uses alongside feature geometry to deepen mechanistic understanding of LMs
Control And Perception Loopframework0.771
Causal importanceconcept0.760
A measure of whether a subcomponent is necessary to reproduce model behavior on a specific prompt, predicted by the causal importance network.