claim

active

claim:bottom-up-interpretability-explains-computation-in-the-model-s-own-terms-rather-than-imposing-top-down-abstractions

Bottom-up interpretability explains computation in the model's own terms rather than imposing top-down abstractions

VPD is positioned as advancing a paradigm shift from top-down mechanistic interpretability (activation-based) to parameter-centric, data-driven discovery.

Source paper

extracted_from

cimcWhitepaper

Neighborhood — ranked by edge-count

Findings (1)

finding

Subnetwork for predicting 'her' vs 'his' in 'the princess lost her crown' involves femaleness signal routing via attention and syntactic role detection
supports
Detailed case study demonstrating how VPD subnetworks can be traced to reveal multiple interpretable computational pathways for a single prediction.

Communities (4)

community

Mechanistic interpretability & model evaluation
members_of
Spans attention head decomposition, benchmark awareness, and genomic pathogenicity prediction via neural models.
Mechanistic interpretability via parameter decomposition
members_of
Tracing information flow through weight matrices and attention heads using attribution graphs to identify causally important subcomponents in language models.
AI phenomenology & mechanistic interpretability
members_of
Linking mechanistic interpretability methods to validating AI self-reports of inner experience
Mechanistic interpretability through parameter analysis
members_of
Understanding neural network computation by examining weights, circuits, and signal routing rather than activation patterns alone.

Claims (1)

claim

Activation-based interpretability does not immediately explain the computations that gave rise to activations; understanding parameters is necessary for deeper insight
extends
Motivates shift from studying model activations ('thoughts') to understanding parameters ('the computations themselves').

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Bottom-up interpretabilityconcept0.859
An interpretability paradigm that explains computation in the model's own terms, rather than imposing top-down abstractions; VPD aims to realize this.
The field of interpretability has focused mainly on understanding model activations, not the computations themselvesclaim0.793
Motivation for VPD's parameter-focused approach.
Interpretive abstraction is a fairly simple programming technique, but it's only possible if the parallelism tools are under the programmer's control.claim0.780
Contrasts with auto-parallelizing compilers; flexibility of Linda.
Interpretability features converge across different model architectures, revealing structural similarities.claim0.775
Causal abstraction is not enough for mechanistic interpretability because it becomes vacuous without assumptions about how models encode informationclaim0.769
Central thesis of the paper
Making information explicit—extracting normal forms from computation.claim0.763
Model precomputes answers before tool invocation and attends to cached answer over tool output when discrepancy exists, confirmed via attribution graphs.finding0.761
Mechanistic insight surfaced by NLA explanations and validated through independent causal attribution method.
NLA explanations can contain claims about the target model's input context that are verifiably false, but are typically thematically faithful to the context.claim0.761
Key limitation identified: NLAs hallucinate specific details while preserving thematic accuracy; informs practical usage.

Cross-corpus bridges (1)

same_concept_as · Nomic cosine

External markdown files that talk about the same concept as this entity.

aboutblank_kb
How do these relationships arise bottom-up without presupposing the higher-level individual?questions/how-do-these-relationships-arise-bottomup-without-presupposing.md0.790