Golden Gate Bridge feature neighborhood includes Alcatraz, Presidio, Lake Tahoe, Yosemite; decoder cosine similarity maps onto semantic relatedness.

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Golden Gate Bridge feature [34M/31164353] fires strongly on Wikipedia snippets in Chinese, Japanese, Korean, Russian, Vietnamese, Greek.finding0.794
  Demonstrates multilingual generalization of SAE features.
• For four example features (Golden Gate Bridge, brain sciences, monuments, transit infrastructure), all strong activations (top bucket) received specificity rating 3 from Claude 3 Opus.finding0.750
  Validation that top activations are highly specific to interpretation.
• The Golden Gate Bridgeconcept0.746
  Example: a bridge whose construction process and sensitive placement intensified the natural beauty of the gap.
• Clamping Golden Gate Bridge feature to 10x max activation caused the model to self-identify as the Golden Gate Bridge.finding0.726
  Strong causal evidence that the feature represents the bridge.
• Feature neighborhood exploration via cosine similarity of decoder weightsmethod0.708
  Identifying related features by cosine distance in SAE decoder space.
• In early layers, LLaMA-2-13B represents a 'close association' feature that correlates with truth on cities but anti-correlates on neg_citiesclaim0.702
  Hypothesized intermediate feature explaining antipodal alignment between cities and neg_cities in early-middle layers
• A 'San Francisco' feature in 1M SAE splits into 11 fine-grained features in 34M SAE.finding0.702
  Empirical observation of feature splitting.
• The geometry of the Marriott hotel in San Francisco could not have arisen from a step-by-step unfolding; it is contrived and lacks naturalness.claim0.700
  Marriott hotel case.