HOT-4: Sparse and smooth coding generating a 'quality space'

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Cluster 4 (4-cluster): SIMPLICITY AND INNER CALM, ECHOES, GRADIENTS, POSITIVE SPACEfinding0.740
  Fourth cluster of the four-cluster grouping, containing properties 14, 12, 10, 5.
• HOT-1: Generative, top-down or noisy perception modulesconcept0.738
  Indicator: multiple sources of perceptual activity including top-down and noise required for reality monitoring.
• Sparse autoencoders produce interpretable features for large models.claim0.726
  Central claim of the paper: the method scales to state-of-the-art transformers.
• Sparse Autoencoderframework0.724
  Interpretability framework used to decompose layer-40 activations into sparse feature sets for studying emotional alignment and persistence
• VPD subcomponents are sparse, interpretable, and avoid feature splitting.claim0.718
  Assertion about the qualitative advantages of VPD's rank-one decomposition.
• Training the sparse autoencoder on more data makes features subjectively sharper and more interpretableclaim0.718
  Empirical principle discovered during autoencoder training; led to using 8 billion training points
• Sparse Autoencoders Find Highly Interpretable Features in Language Models (Cunningham et al., 2023)concept0.718
  Core methodology paper for SAE-based interpretable feature extraction
• Scaling laws can be used to guide the training of sparse autoencoders.claim0.717
  Compute-optimal hyperparameters follow predictable power-law relationships.