claim

active

claim:probe-based-data-attribution-bridges-interpretability-work-probes-activations-with-data-centric-alignment

Probe-based data attribution bridges interpretability work (probes/activations) with data-centric alignment.

Conceptual framing: integrates mechanistic interpretability tools with alignment-focused data curation.

Source paper

extracted_from

Probe-Based Data Attribution: Surfacing and Mitigating Undesirable Behaviors in LLM Post-Training

(2026) · Frank Xiao · Santiago Aranguri

Neighborhood — ranked by edge-count

Communities (4)

community

Mechanistic interpretability & model evaluation
members_of
Spans attention head decomposition, benchmark awareness, and genomic pathogenicity prediction via neural models.
Mechanistic interpretability via parameter decomposition
members_of
Tracing information flow through weight matrices and attention heads using attribution graphs to identify causally important subcomponents in language models.
Probe-based training data attribution
members_of
Uses linear probes on activations to identify and filter harmful training data cheaply (~$30).
Activation-based interpretability and mechanistic grounding
members_of
Uses probes, activation patching, and mechanistic analysis to ground abstract concepts in model computations, bridging interpretability with data-centric alignment.

Concepts (2)

concept

Activation space
cites
Representation space on which linear probes operate to attribute harmful behaviors to training data.
Data-centric machine learning
cites
Approach emphasizing data quality and source identification rather than only model architecture changes.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Probe-based method bridges interpretability (probes/activations) with data-centric alignment workclaim0.884
Assertion from the paper's notes that the work connects two previously separate areas: interpretability tools and data-centric alignment.
Probe-based data attribution for alignmentconcept0.872
Probe-based data attribution effectively reduces harmful behaviors via data interventionsclaim0.860
Authors' central interpretive assertion that their method meaningfully mitigates unwanted behaviors.
Probe-Based Data Attributionmethod0.841
Linear classifier approach applied to model activations to identify which training datapoints caused undesired behaviors in post-training.
Are high-accuracy probe representations also causally relevant for the task?question0.785
Question raised by the discrepancy between DAS IIA and linear probe accuracy in Case Study II
A probe may achieve high performance even on representations that are not causally relevant for the taskclaim0.784
Key interpretive claim from Case Study II distinguishing probe accuracy from causal relevance
Direct probes over learned activations in standard basis may fail to reveal the actual causal role of representations because they are highly distributedclaim0.775
Supported by the finding that non-trivial rotations are required to find aligned representations.
Probes trained under different explicit instruction variants are highly aligned with each other despite different wording.claim0.772
Shows the key divide is passive vs. active framing, not the specific wording of instructions.