finding

active

finding:high-low-frequency-detectors-found-across-alexnet-inceptionv1-vgg19-and-resnetv2-50

High-low frequency detectors found across AlexNet, InceptionV1, VGG19, and ResNetV2-50

Second low-level feature type demonstrating cross-architecture universality

Source paper

extracted_from

Zoom In: An Introduction to Circuits

(2020) · Chris Olah · Nick Cammarata · Ludwig Schubert · Gabriel Goh +2

Neighborhood — ranked by edge-count

Claims (1)

claim

Analogous features and circuits form across models and tasks.
supports
Third of three speculative claims asserting that learned features are not model-specific but represent universal solutions to learning problems

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Curve detectors found across AlexNet, InceptionV1, VGG19, ResNetV2-50 and models trained on Places365finding0.861
Anecdotal evidence for the universality of low-level visual features across different architectures and datasets
We hypothesize that high-low frequency detectors, if predicted by artificial neural network universality, might be found in biological neural networks.hypothesis0.820
Specific cross-domain prediction mentioned by neuroscientists in conversation with the authors
High-Low Frequency Detectorconcept0.778
A less intuitive feature family detecting low-frequency patterns on one side of the receptive field and high-frequency on the other; used as example of non-obvious but understandable features
InceptionV1 spreads car feature from a pure car detector in mixed4c across dog detector neurons in the next layerfinding0.739
Circuit-level evidence that polysemantic neurons arise deliberately through superposition rather than entangled computation
InceptionV1 implements a four-layer circuit for pose-invariant dog head detection with mirrored left/right pathways that inhibit each other then unite, exhibiting XOR-like propertiesfinding0.738
Evidence that neural networks learn sophisticated invariance mechanisms through structured circuits rather than loose feature aggregation
Weights between early and full curve detectors in InceptionV1 form a curve of positive weights at tangent positions, with opposing orientations inhibitoryfinding0.737
Demonstrates that meaningful algorithms can be read directly off floating-point weights in a neural network
26 candidate off-topic detector latents identified in Llama-3.3-70B via contrastive searchfinding0.733
Core mechanistic finding identifying specific SAE latents associated with ESR
Net detection signal (detection minus control) is near-zero across all 40 layer-strength configurations: mean = -0.01 ± 0.03 logitsfinding0.703
Quantitative evidence that binary detection provides no genuine introspection signal beyond global logit shifts