finding

active

finding:concept-interventions-on-some-concepts-act-as-wrecking-ball-interventions-collapsing-global-model-performance

Concept interventions on some concepts act as 'wrecking-ball' interventions, collapsing global model performance.

Observation of catastrophic performance drop when steering certain concepts.

Source paper

extracted_from

Mechanistic Interpretability of EEG Foundation Models via Sparse Autoencoders

(2026) · William Lehn-Schiøler · Magnus Ruud Kjær · Rahul Thapa · M. Pedersen +9

Neighborhood — ranked by edge-count

Claims (1)

claim

Some SAE concept steering interventions act as 'wrecking balls' that collapse global model performance rather than selectively modifying target concepts.
supports
A critical failure mode identified in the paper demonstrating risk of naïve concept steering

Communities (3)

community

Manifold-aware concept steering in neural representations
members_of
Explores geometry of activation/behavior manifolds to enable selective, non-destructive concept interventions.
Geometric concept representations in neural networks
members_of
Concepts encoded as curved manifolds and circular structures in LLM activation spaces.
Concept steering & catastrophic model failure
members_of
Studies how targeted interventions on learned concepts can cause sudden, global collapse in neural network performance.

Questions (1)

question

Can clinical concepts be selectively steered without damaging unrelated performance?
answered_by
Question about the feasibility of safe concept steering in EEG models.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

wrecking-ball interventions that collapse global model performancequote0.890
Load-bearing phrase describing catastrophic steering effects.
Wrecking-ball interventions that collapse global model performance are empirically identified in EEG foundation models.finding0.861
Demonstrates a critical failure mode of concept steering with clinical safety implications
Can concept steering interventions on EEG foundation models be made selective rather than globally destructive?question0.771
Research question motivating the introduction of the probe area metric and identification of operational regimes
wrecking-ball interventionconcept0.762
Type of concept steering intervention that catastrophically collapses global model performance.
Piggybacking a new purpose onto an existing concept (overloading) causes conflicts and design flaws.claim0.743
Illustrated with OS X print subsystem example.
Existing intervention libraries are often project-based, lack extensibility, are hard to maintain and share, and are limited to single or non-nested interventions on Transformersclaim0.739
Motivation claim contrasting pyvene with prior tools like BauKit, TransformerLens, nnsight, graphpatch
Intervention on a balanced subspace dimension while holding others fixed crosses the decision boundary using a non-native mechanismfinding0.739
Additional synthetic example of pernicious divergence from balanced subspaces
Representation engineering for large-language models: Survey and research challenges (Bartoszcze et al., 2025)concept0.737
Survey of representation engineering methods cited as related work