Contrastive SAE Training Procedure

Procedure mapping hidden representations into SAE space and applying contrastive loss to learn facet-aligned control vectors

Neighborhood — ranked by edge-count

paper

method

Prototype Contrast Loss (L_CE)
uses
Loss function pulling representations toward positive centroid and pushing away from negative centroid with angular margins
F-statistics and Linear Probes for Feature Selection
uses
Method to select d_steer top-activated SAE features for constructing control vectors
Positive Pull Loss (L_dist)
uses
Distance-based loss comparing injected representation to class centroids in the active subspace
Residual Stream CV Injection
uses
Technique of adding control vectors to model hidden states at mid-residual layers without weight updates

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Contrastive learningframework0.767
Supervised learning framework where system learns by observing contrast between current response and nudged improved response; requires weak additional forces from supervisor
Contrastive Sparse AutoEncoder (CV-SAE) Frameworkframework0.752
The primary novel framework introduced in the paper for learning facet-level personality control vectors
Contrastive Stimulus Designconcept0.735
LAT methodology step constructing paired prompts that elicit divergent behaviors to extract steering vectors
SAE features trained on text activations generalize to image inputs, activating on relevant visual depictions.finding0.730
Out-of-distribution generalization of SAE features.
SAE features generalize to images despite training only on text, indicating out-of-distribution robustness.claim0.727
A promising property for interpretability analysis off-distribution.
Surgical Trainingconcept0.727
Training approach targeting only functionally specialized components to avoid catastrophic forgetting and misalignment
Contrastive analysismethod0.726
Method comparing brain activity in conscious vs. unconscious conditions.
Contrastconcept0.718
The property that living structures contain intense contrast—far more than one imagines helpful; true opposites which annihilate each other when superimposed, creating differentiation that gives birth to something; contrast unifies rather than separates when used correctly