finding

active

finding:weights-between-early-and-full-curve-detectors-in-inceptionv1-form-a-curve-of-positive-weights-at-tangent-positions-with-opposing-orientations-inhibitory

Weights between early and full curve detectors in InceptionV1 form a curve of positive weights at tangent positions, with opposing orientations inhibitory

Demonstrates that meaningful algorithms can be read directly off floating-point weights in a neural network

Source paper

extracted_from

Zoom In: An Introduction to Circuits

(2020) · Chris Olah · Nick Cammarata · Ludwig Schubert · Gabriel Goh +2

Neighborhood — ranked by edge-count

Claims (2)

claim

Features are connected by weights forming circuits, and these circuits can be rigorously studied and understood as meaningful algorithms.
supports
Second of three speculative claims asserting that subgraphs of neural networks are tractable and meaningful objects of study
Individual floating-point number weights in neural networks become meaningful once you understand the features they connect.
supports
Interpretive claim that circuits render raw weights interpretable as algorithmic structures

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Curve detectors found across AlexNet, InceptionV1, VGG19, ResNetV2-50 and models trained on Places365finding0.813
Anecdotal evidence for the universality of low-level visual features across different architectures and datasets
InceptionV1 spreads car feature from a pure car detector in mixed4c across dog detector neurons in the next layerfinding0.783
Circuit-level evidence that polysemantic neurons arise deliberately through superposition rather than entangled computation
InceptionV1 implements a four-layer circuit for pose-invariant dog head detection with mirrored left/right pathways that inhibit each other then unite, exhibiting XOR-like propertiesfinding0.760
Evidence that neural networks learn sophisticated invariance mechanisms through structured circuits rather than loose feature aggregation
InceptionV1 neuron 4e:55 responds to cat faces, fronts of cars, and cat legs as unrelated stimulifinding0.750
Concrete example of polysemantic neuron demonstrating the challenge to the circuits agenda
High-low frequency detectors found across AlexNet, InceptionV1, VGG19, and ResNetV2-50finding0.737
Second low-level feature type demonstrating cross-architecture universality
The experimental method of starting with a vision and adding color to create life in the centers has predictable results: the eleven invariants.claim0.731
Assertion that the process yields a specific set of color qualities, listed in the chapter.
Steering vectors discover effective triggers such as 'However' and 'Otherwise', consistent with prior reported reflection datasetsfinding0.720
Validates that steering vectors capture reflection semantics by finding tokens reported in related work.
In the Mountain Car case study, car position is a 1D manifold; linear interventions cross voids causing incoherence; following the 1D curve produces smooth control.finding0.719
Empirical demonstration that a semantically meaningful variable is encoded as a curved manifold, and that respecting its geometry is critical for effective intervention.