finding
active
finding:attribution-graph-tracing-information-flow-across-parameter-subcomponents-for-specific-model-predictions-e-g-her-vs-his-pronoun-selectionAttribution graph tracing information flow across parameter subcomponents for specific model predictions (e.g., 'her' vs 'his' pronoun selection)
Shows how VPD-identified subnetworks can be analyzed to reveal interpretable pathways of computation (e.g., gender signal routing, syntactic role detection).
Source paper
extracted_fromNeighborhood — ranked by edge-count
Communities (4)
community
- Spans attention head decomposition, benchmark awareness, and genomic pathogenicity prediction via neural models.
- Tracing information flow through weight matrices and attention heads using attribution graphs to identify causally important subcomponents in language models.
- Tracing information flow through parameter subcomponents to isolate computational mechanisms for specific model predictions, using tools like attribution graphs and VPD.
- Mechanistic tracing of information flow through attention and MLP subcomponents for pronoun prediction tasks
Methods (1)
method
- Core technique introduced in this paper for decomposing neural network weight matrices into mechanistically simple, interpretable rank-one subcomponents.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Gradient-based technique using SAE features to estimate causal effects on completions; used to corroborate NLA findings.
- Attribution graph reveals a pathway that detects the verb 'lost' and upweights object pronounsfinding0.793Second component of the subnetwork for 'her', complementing the femaleness signal.
- Detailed case study demonstrating how VPD subnetworks can be traced to reveal multiple interpretable computational pathways for a single prediction.
- Mechanistic insight surfaced by NLA explanations and validated through independent causal attribution method.
- Method to trace how parameter subcomponents interact from input to output for a given next-token prediction, producing a subnetwork graph.
- One component of the minimal subnetwork for predicting 'her', discovered via VPD attribution graph.
- Baseline method against which probe-based ranking is compared; more computationally expensive.
- Feature attribution (gradient-based) correlates 0.8 with ablation effects on the 'John' and 'Kobe' examples.finding0.744Validation of attribution as a fast proxy for causal importance.