Contrastive learning is necessary for aligning latent representations with intended trait polarity; distance-only loss is insufficient and counterproductive

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• A family of contrastive learners converges to a representation whose kernel is the pointwise mutual information (PMI) of the underlying eventshypothesis0.783
  Mathematical formalization of what representation models converge to
• Contrastive learningframework0.762
  Supervised learning framework where system learns by observing contrast between current response and nudged improved response; requires weak additional forces from supervisor
• Latent methods lack task generalization and are difficult to train with autoregressive parallelization.claim0.747
  Identifies key limitations of latent methods.
• Dictionary learning offers advantages over linear probes: amortization of cost and unsupervised discovery of abstractions.claim0.743
  SAE features can be found without pre-specified concepts, and feature steering often outperforms few-shot probe vectors.
• Reinforcement learning can be regarded as a limiting or special case of model-based approaches in general — or active inference in particular — when epistemic value is removed.claim0.742
  §3 Discussion.
• The contrast vector method is recommended over PC1 for reproducing the Assistant Axis in different models because it is not guaranteed that PC1 in every model will correspond to an Assistant Axisclaim0.740
  Practical methodological recommendation based on Llama 3.1 70B failure case
• Reinforcement learning acting on individual characteristics affecting their connections to others can result in dynamics that are equivalent to unsupervised learning at the system scale.claim0.737
  Key insight linking individual rewards to system-level learning.
• For future powerful models, situational awareness might 'corrupt' the deployment representation in contrastive prompts, potentially rendering contrastive steering ineffectivehypothesis0.734
  Future threat to the method: a highly sophisticated model might be suspicious of deployment-framed prompts during extraction.